key: cord-0024251-c5gdx137 authors: Javidnia, Monica; Arbatti, Lakshmi; Hosamath, Abhishek; Eberly, Shirley W.; Oakes, David; Shoulson, Ira title: Predictive Value of Verbatim Parkinson’s Disease Patient-Reported Symptoms of Postural Instability and Falling date: 2021-10-12 journal: nan DOI: 10.3233/jpd-212636 sha: 832c9d06a7c13504b2e397d33a690fdc11b60117 doc_id: 24251 cord_uid: c5gdx137 BACKGROUND: Postural instability is an intractable sign of Parkinson’s disease, associated with poor disease prognosis, fall risk, and decreased quality of life. OBJECTIVE: 1) Characterize verbatim reports of postural instability and associated symptoms (gait disorder, balance, falling, freezing, and posture), 2) compare reports with responses to three pre-specified questions from Part II of the Movement Disorder Society Unified Parkinson Disease Rating Scale (MDS-UPDRS), and 3) examine postural instability symptoms and MDS-UPDRS responses as predictors of future falls. METHODS: Fox Insight research participants reported their problems attributed to PD in their own words using the Parkinson Disease Patient Reports of Problems (PD-PROP). Natural language processing, clinical curation, and data mining techniques were applied to classify text into problem domains and clinically-curated symptoms. Baseline postural instability symptoms were mapped to MDS-UPDRS questions 2.11–2.13. T-tests and chi-square tests were used to compare postural instability reporters and non-reporters, and Cochran-Armitage trend tests were used to evaluate associations between PD-PROP and MDS-UPDRS responses; survival methods were utilized to evaluate the predictive utility of PD-PROP and MDS-UPDRS responses in time-to-fall analyses. RESULTS: Of participants within 10 years of PD diagnosis, 9,692 (56.0%) reported postural instability symptoms referable to gait unsteadiness, balance, falling, freezing, or posture at baseline. Postural instability symptoms were significantly associated with patient-reported measures from the MDS-UPDRS questions. Balance problems reported on PD-PROP and MDS-UPDRS 2.11–2.13 measures were predictive of future falls. CONCLUSION: Verbatim-reported problems captured by the PD-PROP and categorized by natural language processing and clinical curation and MDS-UPDRS responses predicted falls. The PD-PROP output was more granular than, and as informative as, the categorical responses. Of the cardinal motor signs of Parkinson's disease (PD), postural instability is the least responsive to dopaminergic therapies [1, 2] . Impairment of postural stability is further associated with poor disease prognosis, leading to increased risk of falls, decreased quality of life, and heightened morbidity and mortality [2] [3] [4] [5] [6] . The U.S. Preventive Services Task Force has placed special emphasis on fall prevention and clinician-involvement in preventive efforts for older adults at increased fall risk [7, 8] . Direct patient reporting may be beneficial for earlier identification of people at risk of falls, particularly in the absence of in-person clinical examination. Fox Insight (https://foxinsight.michaeljfox.org) is an online, observational, longitudinal clinical study sponsored by the Michael J. Fox Foundation for Parkinson's Research (MJFF) [9] . Consenting research participants were recruited online and invited to complete initial and follow-up study visits, comprising a series of questionnaires and modules. Within the Fox Insight study is the Parkinson Disease Patient Reports of Problems (PD-PROP) research module developed by Grey Matter Technologies. PD participants reply to two open-ended questions: 1) What is the most bothersome problem for you due to your Parkinson's disease? 2) In what way does this problem bother you by affecting your everyday functioning or ability to accomplish what needs to be done? Respondents report up to their 5 th most bothersome problem and assign a severity score to each problem, ranging on a scale from 0 (not particularly severe) to 3 (very severe). To accrue longitudinal data, participants were invited at about six-month intervals to repeat the PD-PROP module and selected categorical questions [9] . This study aimed to characterize postural instability responses using the PD-PROP, compare the PD-PROP responses to postural instability-related ordinal responses provided by the same subjects at the same study visit through the MDS-UPDRS Part II questions 2.11, 2.12, and 2.13, and determine if the PD-PROP could predict future falls. Fox Insight study participants who had completed a baseline PD-PROP were identified (n = 23,659, data cut February 2020), and 20,246 participants were selected within the first 10 years since PD diagnosis, age < 100 years, and age at diagnosis > 15 years. For the 18,800 participants who completed both PD-PROP and the three MDS-UPDRS Part II questions (2.11, 2.12, 2.13), 17,297 (92%) participants were selected who completed the questions unassisted. For the time-to-falls analysis, the data were further subset to participants who had a minimum of two PD-PROP 'visits', spanning at least 24 months, and who completed MDS-UPDRS Part II and fall reporting data (n = 2,237). Participants who reported falling at baseline in the Non-Motor Symptoms Questionnaire (NMSQ) were further filtered out, leaving n = 1,557 for analysis. Participant selection is summarized in Supplementary Figure 1 . Keyboard-entered verbatim reports from participants were preprocessed using standard natural language processing and data analytics techniques previously described [10] [11] [12] [13] [14] [15] to generate a subset of the PROP dataset for curation and classification of the reports into PD clinical domains and symptoms. The verbatim text was reviewed and curated by three clinicians, two non-clinical researchers, and a PD patient 'experience-expert' who independently reviewed randomly selected samples of text and in turn categorized and labelled the concatenated problem and functional consequence pair under umbrella domains of 'Autonomic Dysfunction', 'Bradykinesia', 'Cognition', 'Fatigue', 'Mood', 'Pain', 'Postural Instability', 'Rigidity', 'Sleep', and 'Tremor'. Within the domains, specific clinically relevant symptoms were similarly curated and categorized. The curated dataset was then used to classify the entire PROP cohort at baseline and across multiple longitudinal visits enabled by a proprietary algorithm using an Apache Lucene [16] phrase query approach For the postural instability domain, five symptom categories of gait disorder, balance, falling, freezing and posture were identified. In order to evaluate the efficiency of the data mining and data extraction process as well as to calculate the specificity and recall of the data extraction algorithm, curators were provided 50 random samples that the machine classified as true positives, and 50 random samples the machine classified as true negatives. The curators were then asked to classify the verbatim based on their knowledge and experience, and the results were then compared back to the machine extracted results. The MDS-UPDRS Part II questions in the Fox Insight 'Your Movement Experiences' questionnaire rate 13 movement-related activities of daily living on a 5-point scale (normal, slight, mild, moderate, severe). Three MDS-UPDRS Part II questions were pre-selected as the most relevant to the postural instability domain and its symptoms: 2.11 (Over the past week, have you usually had trouble getting out of a bed, a car seat, or a deep chair?), 2.12 (Over the past week, have you usually had problems with balance and walking?), and 2.13 (Over the past week, on your usual day when walking, do you suddenly stop or freeze as if your feet are stuck to the floor?). Timeto-falling was analyzed using the no-yes responses to the NMSQ question 21 (Fox Insight Non Movement Experiences Question "Have you experienced falling in the last month?".) Demographic data obtained at Fox Insight enrollment were compared between participants who reported postural instability symptoms and those who did not, using a two-tailed, t-test with Welch's correction or chi-square test, as appropriate. The association between MDS-UPDRS Part II items and reports of the postural instability domain and its symptoms was determined by two-sided Cochran-Armitage trend test. These analyses were performed using Graph-Pad Prism 8.4. Figures were generated with either GraphPad Prism 8.4 or UpSetR. Participants with PROP data spanning at least 24 months, including at least two corresponding NM SQ visits and three MDS-UPDRS II questions at the baseline PROP were selected for the time-tofalling analysis. Separate Kaplan-Meier survivorship analyses of time-to-falling were performed for each baseline PROP symptom stratified by priority and for each MDS-UPDRS question stratified by impairment severity; statistical significance of the priority and impairment ratings was determined using unadjusted Cox proportional hazards models with the not-reported (PD-PROP) and unimpaired (MDS-UPDRS) groups omitted. A multi-variate Cox proportional hazards model to predict time-tofalling assessed the predictive value of the baseline PROP and MDS-UPDRS variables; baseline age (dichotomized at 65), sex and participant-reported years from PD diagnosis (< 3, 3-5, > 5) were included in the models. Five PD-PROP baseline variables were included in the models as reported or not reported. The baseline MDS-UPDRS 2.11-2.13 variables were dichotomized as 0 (no impairment) vs 1-4 (any impairment). The analysis included a selection process to identify the most predictive variables, but age, sex and years from diagnosis were specified to remain in the model. These analyses were performed in SAS (9.4). Data reported in this manuscript are available to qualified researchers through the Fox Insight Data Exploration Network (Fox DEN) at https:// foxden.michaeljfox.org. Independent curator classification of the postural instability domain and its symptoms showed high validation characteristics. Postural instability symptoms averaged 95.1% specificity and 99.3% recall/sensitivity (Supplementary Table 1 ). A total of 17,297 Fox Insight participants are included in these analyses. Demographic features Values shown are the mean, standard deviation, and range. The p-value was determined between postural instability reporters and non-reporters by two-tailed t test with Welch's correction for age and years since diagnosis and chi-square test for sex. are detailed in Table 1 . The 9,692 (56.0%) participants who reported problems associated with postural instability at baseline were significantly older, had a longer time from diagnosis and were more likely to be female than those who were not. Symptom reports in the postural instability domain were; gait disorder (n = 6,800, 70.2% of those who reported postural instability), balance (n = 4,213, 43.5%), falling (n = 2,711, 28.0%), freezing (n = 762, 7.9%), and posture (n = 489, 5.0%). Figure 1 shows whether symptoms were reported singly or in combination with other symptoms. Priority and severity of each postural instability symptom are detailed in Table 2 . Symptoms were reported most often with a severity of 1 or 2 on a least-most scale of 0-3. The domain of postural instability and its symptom reports were consistently and significantly associated Priority and severity of postural instability (overall) and associated symptoms among 9,692 postural instability-reporting participants at their baseline PD-PROP completion. Postural instability and its symptoms were most frequently reported as a first most bothersome problem. Severity was predominantly rated as 1 or 2 on a least-most scale of 0-3. postural instability symptoms]) were generated, with a darker color indicating a higher number of participants' intersecting responses. The highest proportions of postural instability responses fell at the intersection of "not reported" on the PD-PROP and "normal" or "slight" on the MDS-UPDRS questions. Some first priority PROP problems were designated as "slight" or even "normal" on the MDS-UPDRS. Data from 1,557 participants were followed longitudinally for time-to-falling analysis. The Kaplan-Meier plots for balance stratified by priority (p = 0.0392 for ranking of priority) and for MDS-UPDRS question 2.12 stratified by impairment severity (p < 0.0001) are shown in Fig. 3 . The order of the priority rankings was not significant for the other four symptoms; the order of the severity rankings was also significant for question 2.11 (p < 0.0001) but not for question 2.13. Using a Cox proportional hazards model to predict time-to-falling (responded 'Yes' to NMSQ Question 21), the most predictive variables comprising the final adjusted model were reporting problems with balance on the PD-PROP, and impairment on MDS-UPDRS questions 2.11-2.13 (Table 3) . Age, female sex, and > 5 years since diagnosis were also significant predictors. Patient-reported verbatim replies to open-ended questions, using validated clinical curation and natural language processing, demonstrated that symptoms in the postural instability domain correlated well with corresponding questions on Part II of the MDS-UPDRS. Both are predictors of future falls. The PD-PROP balance symptom contributed independently to the risk of falling. The finding that female participants with PD report falling more frequently than male participants has been reported among the predictors of progression to falling in PD [17, 18] . The explanation for this observation may be due to greater reporting tendencies among female participants; however, the confirmation of this sex difference in Fox Insight by both PD-PROP and UPDRS part II deserves further exploration. PD genotypes have also been shown to contribute to the onset of postural instability [19] , prompting future research to examine the effects of genotype on patient-report PD-PROP symptoms of postural instability. As the Fox Insight study progresses and more longitudinal data are available, clinically-meaningful events beyond 24 months will be examined. Additional 'deep phenotyping' efforts of PD are also underway, to relate what patients say in the PD-PROP to what they do using digital monitoring measures [20] . Both the MDS-UPDRS Part II and PD-PROP participant responses represent categorical measures and quantitative output. The PD-PROP requires more time and effort at the front end of patient reporting, but once the curation and classification have been completed and the data dictionary compiled, this approach of analyzing verbatim text can be applied more widely and produce more granular and richer data. The MDS-UPDRS question 2.12 is indeed a strong predictor of falls, but otherwise provides limited information. The key methodological difference is that the MDS-UPDRS Part II categorization is locked within a 0-4 ordinal scale while the PD-PROP classification is based on verbatim replies to open-ended questions about reported problems and their functional consequences, and elaborated further by self-reported priority of the problem (from most bothersome) and the severity of the problem. Further, PD-PROP has the additional advantages of 1) being electronically transcribed, eliminating the need to transcribe from paper to electronic formats, and improving data accuracy [21] , 2) capturing patient priorities and functional consequences, and 3) being suited for re-curation (e.g., separating falling from fear of falling). In the longitudinal context of PD randomized controlled trials, the MDS-UPDRS Part II has shown little change over time in early PD and so may not be ideal to quantify disease progression [22, 23] . The PD-PROP may be more informative but has not yet been applied to longitudinal interventional trials. Natural language processing and clinical curation could also be applied more widely to clinical care in the form of analyzable patient or clinician text to openended questions. The MDS-UPDRS Part II and PD-PROP are both predictive of falls and therefore have value in foretelling the occurrence of a serious outcome, such as falls that portend morbidity and mortality [2] [3] [4] [5] [6] . Their predictive value for falling could inform the design of clinical trials aimed at forestalling the occurrence of this clinically relevant outcome, a finding that would be tantamount to disease modification since existing treatments have not been demonstrated to exert therapeutic benefit on the clinical domain of postural instability, the symptom of balance, or the occurrence of falling, The Fox Insight longitudinal data, accessible to researchers via Fox DEN (https://foxden.michael jfox.org), are entirely reported by the participant (or care partner), without investigator assessment or input. In the current COVID-19 era of burgeoning telemedicine and virtual clinical trials, there is lingering concern about examination-related limitations of sole patient reporting, particularly with regard to PD signs of rigidity and postural reflexes [24] . Regardless, patient-reported problems, functional consequences and derived symptoms remain relevant for clinical research and care. The case can be made that what the patient says is as or more important than what the clinician hears or observes [25, 26] . Influence of dopaminergic medication on automatic postural responses and balance impairment in Parkinson's disease Impact of falls and fear of falling on healthrelated quality of life in patients with Parkinson's disease Variable expression of Parkinson's disease: A baseline analysis of the DATATOP cohort. The Parkinson Study Group Quality of life in Parkinson's disease: The relative importance of the symptoms Falls in Parkinson's disease. Causes and impact on patients' quality of life Clinical and physiological assessments for elucidating falls risk in Parkinson's disease Interventions to prevent falls in community-dwelling older adults: US Preventive Services Task Force Recommendation Statement Prevention of falls in older adults Fox Insight collects online, longitudinal patient-reported outcomes and genetic data on Parkinson's disease The Apache Software Foundation NLTK 3.5 documentation, NLTK Project Strength Natural Language Processing In Python, spaCy The Apache Software Foundation What predicts falls in Parkinson disease?: Observations from the Parkinson's Foundation registry Factors associated with falling in early, treated Parkinson's disease: The NET-PD LS1 cohort When does postural instability appear in monogenic parkinsonisms? An individual-patient meta-analysis Deep phenotyping of Parkinson's disease Inconsistent Movement Disorders Society-Unified Parkinson's Disease Rating Scale Part III Ratings in the Parkinson's Progression Marker Initiative Progression of MDS-UPDRS scores over five years in de novo Parkinson disease from the Parkinson's Progression Markers Initiative Cohort Does the MDS-UPDRS provide the precision to assess progression in early Parkinson's disease? Learnings from the Parkinson's progression marker initiative cohort Movement Disorder Society-Unified Parkinson's Disease Rating Scale use in the Covid-19 era What patients say Patient-Focused Drug Development: Methods to Identify What Is Important to Patients. Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders (Draft Guidance) The authors thank Amy Amara, MD, PhD, Connie Marras, MD, PhD, David Standaert, MD, PhD, and Caroline Tanner, MD, PhD for their expert clinical curation of verbatim reports in Fox Insight and Andrew Nguyen PhD for his technical support. We extend our special gratitude to the thousands of Fox Insight research participants and to the MJFF for supporting and enabling this remarkable online research study.Research The supplementary material is available in the electronic version of this article: https://dx.doi.org/ 10.3233/JPD-212636.