Several recent publications described algorithms to identify subjects with Parkinson’s disease (PD). In creating the “PREDIGT Score”, we previously developed a hypothesis-driven, simple-to-use formula to potentially calculate the incidence of PD. Here, we tested its performance in the ‘De Novo Parkinson Study’ (DeNoPa) and ‘Parkinson’s Progression Marker Initiative’ (PPMI); the latter included participants from the ‘FOllow Up persons with Neurologic Disease’ (FOUND) cohort. Baseline data from 563 newly diagnosed PD patients and 306 healthy control subjects were evaluated. Based on 13 variables, the original PREDIGT Score identified recently diagnosed PD patients in the DeNoPa, PPMI + FOUND and the pooled cohorts with area-under-the-curve (AUC) values of 0.88 (95% CI 0.83–0.92), 0.79 (95% CI 0.72–0.85), and 0.84 (95% CI 0.8–0.88), respectively. A simplified version (8 variables) generated AUC values of 0.92 (95% CI 0.89–0.95), 0.84 (95% CI 0.81–0.87), and 0.87 (0.84–0.89) in the DeNoPa, PPMI, and the pooled cohorts, respectively. In a two-step, screening-type approach, self-reported answers to a questionnaire (step 1) distinguished PD patients from controls with an AUC of 0.81 (95% CI 0.75–0.86). Adding a single, objective test (Step 2) further improved classification. Among seven biological markers explored, hyposmia was the most informative. The composite AUC value measured 0.9 (95% CI 0.88–0.91) in DeNoPa and 0.89 (95% CI 0.84–0.94) in PPMI. These results reveal a robust performance of the original PREDIGT Score to distinguish newly diagnosed PD patients from controls in two established cohorts. We also demonstrate the formula’s potential applicability to enriching for PD subjects in a population screening-type approach.
Parkinson’s disease (PD) remains an incurable neurodegenerative condition. The worldwide prevalence of PD is estimated at 6.1 million1 and its incidence is expected to double by 20302. As of 2022, no disease-modifying agents have been approved and no preventive therapy exists to delay the onset of clinical PD in those at higher risk. It is widely assumed that an earlier diagnosis of PD, including in its prodromal phase, would enable future breakthroughs in disease modification3. To this end, establishing a reliable working diagnosis early in its course appears essential.
Predicting the future incidence of PD in neurologically healthy individuals is challenging due to the incomplete understanding of the disease’s aetiology and the limited availability of large, prospective cohorts of neurologically healthy individuals followed for incident PD. Recently, a small number of models have been developed using data analyses-driven approaches to identify subjects in the earliest stages of PD4,5,6,7,8,9,10,11,12. Berg et al. presented a three-step approach to determine the risk of incident PD4. The approach by Nalls et al.5 distinguished PD patients from healthy controls (HC) in the ‘Parkinson’s Progression Marker Initiative’ (PPMI) cohort with an area-under-the-curve (AUC) value of 0.92. External validation of this model showed good classification of PD (AUC ≥0.9). In the Schrag et al. approach6, neurological and psychiatric variables, such as tremor, rigidity, cognitive decline and depression, were integrated together with autonomic variables. This model was recently refined to enhance the prediction algorithm’s performance7.
On behalf of the International Movement Disorder Society (MDS), Berg et al. proposed an algorithm to diagnose subjects in a prodromal stage of PD8,9. The MDS Research Criteria have since been validated in three prospective studies: an elderly community-based cohort10, a longitudinal cohort of REM sleep behavior disorder (RBD) subjects11, and a genetically characterized cohort of mutant LRRK2 allele carriers12.
In parallel, we had designed the PREDIGT Score model as a hypothesis-driven approach to quantify disease incidence based on established risk factors linked to the pathogenesis of PD13. In this approach, the selection of variables and their coefficients was not restricted to any particular dataset (or a specific cohort). Instead, the PREDIGT model was based on: (1) the hypothesis that PD is a complex disorder requiring interactions between several risk elements to promote disease; (2) an umbrella review of 75 meta-analyses to estimate the effect size of each identified variable; and (3) a model for pathogenesis that sees the gradual evolution from a healthy state to a prodromal phase on to the manifestation of cardinal signs, which subsequently lead to the diagnosis of PD14.
We had postulated that the development of ‘idiopathic’ PD could be explained by interacting contributions from five risk categories: Exposome, as the sum of environmental exposures (E); genetic susceptibility (DNA variants; D); the presence of tissue changes from documented gene-environment Interactions (I); sex/Gender (G); and age, as in the passage of Time (T). We had further proposed -and mathematically modeled- that a final risk score could be calculated by assigning values to each category, as computed by the simple formula of: PR = (E + D + I) x G x T. For select risk elements, where details of variables remain unknown, such as for a person’s exposome history (E) within the nasal cavity and gastrointestinal tract, or for the specifics of inherited genetic variants (D), we had identified simple-to-impute surrogates, such as hyposmia, constipation, and family history, respectively13. Motor assessment-based data from validated questionnaires and neurological examinations, such as captured by the UPDRS subscales, were not included in the original model. The reason for this decision -when creating the model- had been the related goal to calculate future PD incidence in neurologically still healthy individuals; this had also been the stated goal of the MDS Research Criteria for Prodromal Parkinson’s Disease effort8,9.
Here, as a first step to evaluate the PREDIGT Score’s performance, we set out to assess its discriminative ability to identify persons that have been diagnosed at an early stage of PD versus HC subjects. Further, we also designed and tested a practical plan to apply the PREDIGT Score to potential population-type screening: In Step 1, we used self-reported answers to a standardized ‘PREDIGT Questionnaire’ for the enrichment of individuals with a higher risk profile, followed in Step 2 by the addition of one objective, clinical measurement from a group of seven biological markers.
In pursuit of these two objectives, we used data from two well-established, previously characterized case-control studies, i.e., the ‘De Novo Parkinson Study’ (DeNoPa)15 and PPMI16. Data from the ‘FOllow Up Persons with Neurologic Disease Study’ (FOUND)17, a follow-up study for a subset of PPMI participants, were used to supplement the environmental exposure variables that had not been collected during the initial enrollment phase of the PPMI study. The mean stage of disease severity in PD patients was documented by the Hoehn and Yahr score at the time of study enrollment (Table 1). Of note, no subjects in the DeNoPa cohort were enrolled in the PPMI (FOUND) study and vice versa.
The PREDIGT model, its associated coefficients for selected variables, and their imputation into the formula (PR = (E + D + I) x G x T), were previously developed and described13. Therefore, both cohorts can be viewed as external validation datasets. A summary of the study workflow, including data preparation and performance assessment, is illustrated in Fig. 1 and detailed in the Methods section. Model 1 permits the examination of the performance for the original version. Model 2 highlights an approach for the implementation of the PREDIGT Score to potential population screening (Fig. 1).
Demographics and participant characteristics
For the initial analysis, we included 245 DeNoPa subjects (PD: n = 135; HC: n = 110) and 624 PPMI individuals (PD: n = 428; HC: n = 196). The FOUND Study contained a subset of PPMI subjects (PD: n = 141; HC: n = 75). Generally, DeNoPa and PPMI participants showed comparable clinical and demographic characteristics (Table 1). Subjects in the PPMI cohort showed a higher percentage of PD patients (68.59%) than did the DeNoPa study (55.1%); in both cohorts, patients were determined to be predominantly at either Hoehn and Yahr stage-1 or −2 (Table 1). The sex ratio was similar across cohorts and disease categorization: approximately two-thirds of participants in each group were males. Median ages and ranges were comparable across cohort, sex, and disease categorization (PD, HC), although participants in the DeNoPa study were slightly older than those in PPMI (by 3 years in median age), and males were slightly older than females (Table 1); these differences were statistically insignificant (p > 0.05). As expected, participants in FOUND showed similar characteristics as those in PPMI.
Performance of the original PREDIGT Score (Model 1)
PPMI + FOUND cohort
Using the 11 variables listed in Fig. 2a at their previously assigned values (Supplementary Table 1) as well as those for gender and age, an individual score for each participant in the PPMI + FOUND cohort was calculated. As illustrated by the score density plot (Fig. 2b), PD patients had a higher PREDIGT Score than subjects in the HC group. The difference in mean scores (±SD) between these two groups was significant (p < 0.0001) at 112.8 (±55.5) and 58.6 (±39.1), respectively. The AUC value for the PREDIGT Score using the PPMI + FOUND cohort measured 0.79 (95% CI 0.72–0.85) (Fig. 2c). The original PREDIGT Score distinguished PD patients from HC subjects in PPMI with a relatively high degree of specificity at 0.92 (95% CI 0.85–0.97), but at a relatively low degree of sensitivity of 0.51 (95% CI 0.43–0.6), using 106.4 as the threshold.
Model 1’, which included eight variables but encompassed available data from all of the 623 participants in PPMI (PD: n = 427; HC: n = 196), was assessed next (Fig. 2a, d, e). There, the AUC value was 0.84 (95% CI 0.81–0.87), with a sensitivity of 0.7 (95% CI 0.65–0.74) and specificity of 0.82 (95% CI 0.77–0.87) using a PREDIGT Score of 68.2 as the optimal cut-off.
Several associations between included variables and PD diagnosis, such as the presence of hyposmia, constipation, a positive family history of PD, depression, anxiety, and RBD were statistically significant (Fig. 2a). Associations of most variables with increased or reduced risk of PD were congruent with the literature with the exception of smoking18. In PPMI + FOUND, a larger percentage of PD patients had a positive smoking history compared to controls, thus apparently contradicting the protective effect described in other studies. Hyposmia was highly informative in distinguishing between PD and HC groups within this cohort. Variables had similar effect sizes for the FOUND subset as for the PPMI cohort, as expected.
Using the 11 variables listed in Fig. 3a and their previously assigned values (Supplementary Table 1) as well as those for gender and age, an individual score for each participant in the DeNoPa cohort was calculated. As illustrated in the score density plot for the DeNoPa cohort (Fig. 3b), PD patients had a higher PREDIGT Score when compared with the HC group, as expected. The mean score (±SD) of the PD group was significantly higher (p < 0.0001) than that of healthy controls at 107.1 (±53.7) versus 36.6 (±38.6), respectively. The AUC value for the PREDIGT Score in the DeNoPa cohort was 0.88 (95% CI 0.83–0.92) (Fig. 3c). The PREDIGT Score distinguished PD patients from HC subjects in the DeNoPa cohort with 0.94 sensitivity (95% CI 0.9–0.98) and 0.67 specificity (95% CI 0.58–0.75), using 39.2 as the threshold.
By comparison, Model 1’, which excluded those environmental exposure variables that had been missing from enrollment forms in the PPMI cohort (see above, also shown in gray in Fig. 3a), was also applied to the DeNoPa cohort (Fig. 3d, e). There, the calculated AUC value was 0.92 (95% CI 0.89–0.95), with 0.84 sensitivity (95% CI 0.78–0.9) and 0.89 specificity (95% CI 0.83–0.95) using a PREDIGT Score of 65.7 as the optimum threshold.
In DeNoPa, many associations between variables from the five PREDIGT Score categories and PD, such as hyposmia, constipation, caffeinated beverage intake, family history of PD, depression, anxiety, and RBD (as diagnosed by a polysomnogram and related questionnaire), were statistically significant (Fig. 3a), as expected from the literature13,19. Among individual variables, the presence of hyposmia, depression, and constipation were particularly informative in distinguishing PD from HC. Of note, results for head trauma and smoking differed in the DeNoPa participants from those reported in the literature, although neither of which was statistically significant.
When the data sets from the PPMI + FOUND and DeNoPa studies were combined in the analysis, the AUCs of Model 1 (PD, n = 260; HC, n = 184) and Model 1’ (PD, n = 553; HC, n = 306) measured 0.84 (95% CI 0.8–0.88) and 0.87 (95% CI 0.84–0.89), respectively.
From these results, we concluded that the original PREDIGT Score discriminated newly diagnosed PD patients with relatively high accuracy from age- and sex-matched healthy individuals in the PPMI and DeNoPa cohorts. Of note, we also examined the model’s performance between male and female subjects using AUC values: within each sex, the discriminative performance of the PREDIGT Score was very similar, i.e., it did not perform significantly better for males or females, in the two cohorts (not shown).
A two-step model for screening purposes (Model 2)
We next explored a population screening-oriented version of the PREDIGT Score (Fig. 4), again relying on data from the DeNoPa and PPMI studies. The data encompassed subjective data from questionnaires (Supplementary Table 3) and results for seven different, objective tests (Supplementary Table 4); these had been administered at the time of each person’s baseline visit according to standardized operating procedures. We did not include results for motoric function.
The PREDIGT Questionnaire (Step 1 of Model 2)
Self-reported answers to questions asked in clinical research forms were selected to construct cohort-specific questionnaires (see Supplementary Table 2 and Supplementary Table 3). The construct derived from the DeNoPa study (13 variables) was first applied to DeNoPa participants, while the questionnaire derived from the PPMI study (12 variables) was tested against both cohorts, i.e., PPMI + FOUND and DeNoPa (Fig. 5). The AUC value for DeNoPa subjects’ responses to the questionnaire in Step 1 was 0.81 (95% CI 0.75–0.86) in differentiating the PD group from HC subjects (Fig. 5a). With fewer questionnaires employed at the time of enrollment, the construct derived from PPMI showed a lesser performance in Step 1 of screening, as expected. When tested against DeNoPa participants, its AUC value was 0.72 (95% CI 0.65–0.78) (Fig. 5b); when tested against PPMI + FOUND participants, its AUC value was 0.64 (95% CI 0.56–0.71) (Fig. 5c). These results highlighted -among others- the importance of including self-reported information on the degree of subjects’ sense of smell, which had been captured at enrollment in the DeNoPa study but had been omitted at the time of enrollment into PPMI.
Determining a threshold for Step 1 in preparation for Step 2
Determining a threshold value in Step 1 (C1) to achieve a high degree of discrimination between groups usually depends on the chosen study design, and in our case, also on the choice of objective marker chosen for Step 2. To aid the development of a screening-oriented variant of the PREDIGT Score (Model 2), we first examined three specific, clinical tests (Fig. 6) in the DeNoPa cohort, namely: the Sniffin’ Sticks performance to quantify hyposmia (a, d); the Beck’s Depression Inventory (BDI) to assess the degree of depression (b, e); and the total α-synuclein concentration in cerebrospinal fluid (CSF) (c, f). Figure 6a–c shows the variable relations between results from the Step 1 questionnaire scores (x-axis) and Step 2 objective assessment scores (y-axis) for the three selected clinical tests. Their corresponding receiver operating characteristic (ROC) curves, as shown Fig. 6d–f, compared the discriminative performances of the Step 1 questionnaire (in black), the Step 2 test by itself (in purple), and the best performing two-step model, i.e., largest combined AUC (in green), to classify individuals as a person with PD or as a healthy control.
Exploring an informative, objective test in Step 2
In the DeNoPa cohort, the Sniffin’ Sticks test results emerged as highly discriminatory in Step 2 (Fig. 6a). Based on objectively measured olfactory function, a high degree of separation of PD patients from HC subjects could be observed both horizontally (Step 1) and vertically (Step 2). The highest combined AUC value was recorded when a cut-off of C1 = 12.48 was applied that advanced 70 (64%) controls and 125 (96%) PD patients into Step 2. There, Model 2 showed an increase in AUC from 0.81 (Step 1; 95% CI 0.75–0.86) to a final AUC value of 0.9 (95% CI 0.88–0.91) when the two steps were combined. When applying a Sniffin’ Sticks test score of 25 (out of a total score of 48) as the cut-off value in Step 2, a sensitivity score of 0.79 and specificity value of 0.94 for the entire DeNoPa cohort were calculated. As shown Figs. 6d and 7, adding a smell test in Step 2 always resulted in a higher combined AUC value than relying on Step 1 alone, regardless of the selected value for C1.
We next tested the performance of results that addressed the presence of depression. In DeNoPa, the separation between HC subjects and PD patients by adding the BDI score, a validated tool for assessing depression, was less informative (Fig. 6b, e) than quantifying olfaction (above); this was due to a rather wide spectrum of BDI scores among PD patients. The largest combined AUC of 0.84 (95% CI 0.80–0.85) was achieved when 38 (35%) control subjects and 110 (83%) patients, each with a Step 1 PREDIGT Score of >34.88, were entered into Step 2. Using the BDI score as the Step 2 test alone, a sensitivity of 0.74 and a specificity of 0.84 were achieved.
Further, when examining the addition of a cerebrospinal fluid (CSF) marker, e.g., the concentration for total α-synuclein, in those subjects that were advanced to Step 2, we saw no improvement in the performance of Step 1 alone in the discrimination of PD from HC groups; rather, it reduced the combined AUC values for several of the selected C1 values. The latter result was attributed to the sizeable overlap in total CSF α-synuclein values between the two groups (Fig. 6c, f), which we and others previously reported20,21,22. In the subset of DeNoPa participants, who had undergone CSF testing (PD: n = 103; HC: n = 65), the largest combined AUC of 0.78 (95% CI 0.75–0.79) was achieved when 72 (70%) PD patients and 17 (26%) HCs, whose Step 1 scores were calculated as >49.5, were advanced into Step 2. The Step 2 threshold was 2.7 ng/ml in CSF α-synuclein concentration, resulting in a sensitivity of 0.66 and specificity of 0.78.
Figure 7 summarizes the comparative results for seven different candidates used in the clinical assessment of non-motor symptoms and objective biological markers in the DeNoPa population (Supplementary Table 4), which were tested in our stepwise Model 2 of the PREDIGT Score. For each test, the corresponding box plot summarizes all the possible combined AUC values when an arbitrary C1 was chosen in Step 1. In the DeNoPa cohort, impaired olfaction, as tested by the Sniffin’ Sticks test, consistently generated the highest combined AUC value among all objective, non-motoric exam- and non-imaging study-based results analyzed. In addition to olfaction, the combined AUC for other clinical tests, such as for the degree of depression, as measured by three different tools, and the presence of a reduction in one of two CSF biomarkers, were also consistently higher than the AUC value calculated from Step 1 alone (Fig. 7).
Similar outcomes for the serial, two-step model were observed when testing seven different, biological variables in the PPMI + FOUND cohort (Supplementary Table 4). Although the Step 1 questionnaire derived from PPMI had not been as informative as its DeNoPa counterpart, with the main reason being that self-reported questions about participants’ sense of smell had been missing (highlighted above in Fig. 5), the combined AUC in differentiating PD from HC subjects was markedly increased when including the results of olfaction testing, such as by UPSIT in the PPMI + FOUND cohort, as Step 2 (Supplementary Figure 1). There, the AUC rose from a value of 0.64 (95% CI 0.56–0.71) to an AUC of 0.89 (95% CI 0.84–0.94) when all the PPMI + FOUND participants were entered into Step 2 and underwent olfaction assessment by UPSIT. When using an UPSIT score of 31 (out of a maximum score of 40) as the Step 2 cut-off, a sensitivity of 0.84 and specificity of 0.85 were achieved. Using other clinical test results in Step 2, such as for a previously reported polygenic risk score5, or the degree of depression, or the presence of anxiety or RBD, or the result of a reduction in select CSF biomarker levels (such as for total α-synuclein or total tau), a small AUC increase was observed in many -but not all- cases (Supplementary Figure 1).
We concluded from these results that in both cohorts, a serial two-step approach, which included the objective assessment of olfaction in Step 2 either by the Sniffin’ Sticks test in DeNoPa or by UPSIT in PPMI + FOUND, achieved the highest degree of accuracy to enrich for subjects with early-stage PD in a screening-type effort, as reflected by combined AUC values of 0.9 and 0.89, respectively. Similar to outcomes in the evaluation of Model 1, the performances of the PREDIGT Score in Model 2 were highly comparable between male and female participants, i.e., we detected no sex difference.
Here, we evaluated the PREDIGT Score in two, well-characterized case-control cohorts. Pertaining to our objective 1, we found that the original formula (Model 1) and its coefficients discriminated recently diagnosed PD patients from sex- and age-matched, healthy individuals with relatively high accuracy using readily accessible data. Notably, this discrimination occurred without reliance on any results for motoric deficits that had been recorded for study participants (Figs. 2, 3).
The unexpected outcome that Model 1’ achieved higher AUC values with fewer variables, when compared to Model 1 (Figs. 2, 3), suggests to us that simplification of the original table of necessary variables could be explored. For example, smoking status could potentially be excluded given our results (Figs. 2a, 3a). In addition, future updates to the list of variables will integrate newly emerging epidemiological and mechanistic insights into the development of PD. There, risk elements, such as well-documented, microbial encounters, confirmed toxicant exposures, and/or other chronic conditions such as inflammatory bowel disease, hypertension, diabetes, and rosacea, will be individually evaluated for possible inclusion23,24,25,26,27. On the protective side, we will analyze cohorts that have quantified physical exercise28.
In the Introduction section, several published algorithms to identify patients with PD, as well as subjects in a prodromal stage, were mentioned. Compared with those models, the PREDIGT Score, which had been designed to test a hypothesis about the pathogenesis of typical PD and to use a readily accessible, simple-to-use formula, performed at similar levels. We posit that both approaches, algorithms developed based on statistical analyses as well as hypothesis-driven models, have advantages and disadvantages. Most likely, each approach will inform the other to improve performance, such as when testing revised versions in the future. Distinct classification methods may also serve different research purposes and clinical applications.
An important insight gained from our study relates to data collection in already ongoing as well as yet-to-be-planned cohort studies. Rigorously standardized research forms and applied methodologies to quantify risk modification across all potential variables, in particular within the growing fields of exposome-, genome-, and biomarker-based research activities, are necessary to better compare outcomes across cohorts. For example, DeNoPa and PPMI employed slightly different definitions of head trauma, which could explain its opposite effects on the association with PD in our univariate analysis (Figs. 2a, 3a). Further, updated research forms should include information on the possible contribution of race, ethnicity, and geography to adequately reflect differences in the worldwide incidence and prevalence rates of PD29.
With respect to the usage of biochemical markers, we noted that while the inclusion of CSF α-synuclein quantification did not significantly improve the performance of the PREDIGT Score in these two cohorts, future integration of results for specific variants, such as of phosphorylated α-synuclein, or for positive signals generated in templating assays in vitro, such as by RT-QuIC measurement, may evolve to be informative30,31,32,33. Similarly, the integration of more refined polygenic risk scores could be tested in the future34,35. Important from the perspective of a movement disorder, in possible revisions of the PREDIGT model both subjective as well as objective motor signs could be integrated in steps 1 and 2, respectively. These could come from different sources, e.g., symptoms reported by subjects, information from wearable devices, and ratings by healthcare practitioners; this, to optimize the identification of subjects at an early stage of already manifest parkinsonism or in a prodromal state with only mild bradykinesia.
The second objective of our study was to examine the PREDIGT Score’s suitability as a simple-to-use, clinical screening tool in the future for the assessment of an individual’s PD risk. To model an enrichment effort for persons at risk of PD in a wider population, or for those who may already be in a prodromal state of PD, we created a variant of the original PREDIGT Score (Model 2); it combines a preliminary score generated from exclusively self-reported information under Step 1 with the results of a single, objective assessment tool in Step 2 (Figs. 4–7). We demonstrate here that the combination of a questionnaire-based score and the result of an objective smell test, as carried out on those individuals with a higher preliminary score in Step 1, generated a robust performance by Model 2 of the PREDIGT Score in distinguishing patients with PD from HC subjects in both cohorts. In practice, the Step 1 questionnaire could be helpful to the (pre-)screening of community-based populations in an inexpensive, unsupervised way to identify possible at-risk individuals for subsequent assessment in Step 2. Here, we focused on the addition of one objective score (rather than multiple tests) in Step 2 for simplicity. An added advantage could be that select tests in step 2, such as simplified smell tests36, could also be performed inexpensively and in an unsupervised manner outside the clinic setting. Alternatively, practitioners could choose several objective tests based on variable study designs. By extension, motor assessments (such as bradykinesia objectively measured) could also be incorporated under Step 2 of the screening-based approach and would likely increase the model’s accuracy.
In sum, we posit that employing a Model 2-like version of the PREDIGT Score could facilitate recruitment of neurologically still healthy persons with higher PD risk into prevention studies, at a time when such measures become available. It could also serve as a tool to screen PD subjects referred to neurologists’ offices for their initial evaluation, beginning with completion of Step 1 at home, followed by completion of Step 2, such as objective assessment of olfaction, at the time of their clinic visit. Moreover, the PREDIGT Score may also aid movement disorder specialists in their efforts to more rapidly recruit homogenous study populations, such as subjects with versus without hyposmia, into clinical trials for manifest PD as well as those in a prodromal state.
For the PREDIGT Score model, selection of variables and their numerical values were based on published meta-analyses previously reported by us13; therefore, the results reported here do not have an over-fitting bias and can be readily tested further using other cohorts and populations. The performance of the PREDIGT Score was evaluated using two well-characterized cohorts in which many (but not all) variables from our original score sheet had been captured. We found that the outcomes were comparable for a single-center study (DeNoPa) vs. a multi-centric (PPMI + FOUND) cohort, and for male vs. female participants. Accuracy of the clinical diagnosis of PD was achieved through ongoing follow-up visits in both cohorts as well as dopamine transporter (DAT) scanning of all enrolled patients in PPMI, thereby permitting robust group classification with the elimination of false positives as well as reliable algorithm assessment. The PREDIGT Questionnaire of Model 2 (in Step 1) is based on self-reported information collected via validated questionnaires that need no involvement by a licensed, clinical practitioner. Thus, an additional, potential benefit of our PREDIGT Score model could be cost effectiveness.
The fact that the PREDIGT Score rests on a hypothesis that utilizes surrogates for several still unknown variables, such as under the category E (exposome), could be seen as a weakness of the original model. We are committed to replacing any surrogate of our current categories with the actual risk modifier in future updates, once they become available, such as from comprehensive microbiome analyses of the nasal cavity and the gastrointestinal tract in validated studies37.
Case-control studies encompass an inherent potential for selection bias in their recruitment. For example, due to the sex- and age-matched study design of the DeNoPa and PPMI, the effect sizes of categories ‘Gender’ and ‘Time’ (age) were not adequately represented. Similarly, details for a positive family history of PD (category D) were also limited in both cohorts’ inclusion/exclusion criteria. Consequently, the performance of the PREDIGT Score reported herein was rather conservative. We will address this limitation in future validation efforts to examine and calibrate coefficients of the PREDIGT Score in actual population settings38. Of note, currently ongoing studies by our team include the testing of the model’s predictive performance using prospective, longitudinal studies of neurologically healthy persons, who were enrolled after having been identified as carrying a specific risk element, e.g., hyposmia or REM sleep abnormalities19,39,40.
To further explore the operational feasibility of the PREDIGT Score model, such as in a two-tiered approach, and to probe for its accuracy to detect PD versus other conditions, we have recently launched a prospective study for its testing at our clinics. There, other neurological disorders, such as Alzheimer’s disease, which also frequently features hyposmia41, atypical parkinsonism, cases of secondary parkinsonism, and forms of dystonia as well as essential tremor, will be included. In planning it, it became apparent that the most expensive part of a PREDIGT Score-based screening approach will be the type of biological marker assessment chosen for Step 2 (invasive, yes/no; healthcare practitioner-administered, yes/no; size of cut-off chosen after Step 1, low/high; etc.).
Therefore, in our work on optimizing the PREDIGT Score and its potential transformation into practice, we are guided not only by similar efforts in the field (e.g.,8,9), recent successes in dementia research42, but foremost by five principles: (i) internal as well as external validation; (ii) simplicity in usage by lay persons and healthcare practitioners; (iii) overall ease of accessibility; (iv) relevance to current health maintenance and future interventions; and (v), last-but-not-least, cost effectiveness.
Source of data and participants
We used de-identified baseline data from subjects enrolled in the DeNoPa and PPMI cohorts. Of note, there was no overlap in participating individuals between the two cohorts. The former15 represents a single-center, observational, longitudinal study of patients with a newly established (“de novo”) diagnosis of PD (UK Brain Bank Criteria43) that were naïve to L-Dopa therapy at baseline, and of age- as well as sex-matched, neurologically healthy controls. Exclusion criteria included previously known or subsequently detected brain conditions, such as normal pressure hydrocephalus, cerebrovascular disease, features of atypical parkinsonism (e.g., multisystem atrophy or progressive supranuclear palsy), and medication-induced parkinsonism. Healthy controls were recruited through relatives and friends of enrolled PD subjects, other patients of the hospital-based clinic as well as through newspaper advertisements in 2009; control subjects had to be without any active, known or previously treated condition of their central nervous system. Diagnostic accuracies for study participants were ensured by ongoing follow-up visits every two years (as of 2022, 10-year follow-up visits are ongoing). Twenty-four patients classified as PD at baseline were later re-grouped as “other neurological diagnosis” (OND) and excluded from the study.
The PPMI cohort16 represents an international, multi-centric (33 centers), observational, longitudinal study. PD patients were diagnosed within the previous two years, confirmed with a DAT scan, and remained untreated. Individuals with other movement disorders were excluded, as described above for DeNoPa participants. Healthy controls had to have no significant neurological dysfunction, no first-degree family member of PD, and a Montreal Cognitive Assessment test score of 26–30 (out of a maximum score of 30). In PPMI, visits took place every three months during the first year and every six to 12 months up to 96 months thereafter. Environmental exposure variables, factor ‘E’ from our model, were not collected in the original PPMI study. To supplement the missing information, data from the FOUND study17 were used, which represents a follow-up study for a subset of PPMI participants; hence, environmental exposure data associated with PD incidence were obtained retrospectively. PPMI participants that fell into other groups (e.g., OND and subjects having ‘scans without evidence of a dopaminergic deficit’ (SWEDD)) were not included in the analysis.
In both cohorts, we focused on previously identified variables13. Data entries related to motor performances were not included in the analysis. Informed consent from all participants was obtained in DeNoPa and PPMI (FOUND) studies. Analyses of their de-identified data sets were approved by the Regulatory Ethics Board of The Ottawa Hospital (20180010-01H).
Procedures and statistical analyses
The published, original PREDIGT Score included 30 variables associated with altered risk for developing PD13. Among those, 20 variables were ascertained at enrollment into the two respective cohorts. After excluding 4 variables with only free-text data entries (e.g., infection history) and low prevalence of exposure (e.g., MPTP-type neurotoxin exposure), we included 16 variables into this study. Most variables had a low rate of missing data (<5%); in such cases, we used corresponding data of the nearest follow-up visit by the same participant for data imputation. CSF biomarker data for total α-synuclein and tau proteins and genetic risk scores (GRS) were excluded from our initial analysis for two reasons: one, they had higher missing data rates (DeNoPa: 32.7%; PPMI: 12.1%); and two, univariate analysis of these three variables showed only mild to moderate association with PD diagnosis (univariate AUC 0.57–0.63). Thus, 13 variables entered phase 1 of our study (Supplementary Table 1).
Environmental exposure variables and family history of PD were self-reported. Multiple instruments to evaluate non-motor symptoms were used in DeNoPa and PPMI (for screening instruments and abbreviations, see Supplementary Table 1). In DeNoPa, the presence of non-motor symptoms was self-reported using the MDS-UPDRS I, Scopa-AUT, PD NMSS, PD NMS, and PDQ-39 questionnaires. MDS-UPDRS I and Scopa-AUT were also used in PPMI for assessing non-motor aspects of PD. Questionnaires for depression (GDS, BDI, MADRS), anxiety (STAI), and REM sleep behavior (RBD SQ) were also included in at least one of the two cohorts. Sniffin’ Sticks test and UPSIT were used to quantify olfaction in DeNoPa and PPMI, respectively. We classified patients as having normosmia, hyposmia, or anosmia by using published cut-off values44,45. Although PPMI participants’ overall genetic risk of having PD were quantified using a previously established, genetic risk score5, detailed genetic risk of PD was not readily available from most of the patients and family members in the clinic; hence, self-reported family history of PD were used as a surrogate for genetic risk information.
Univariate analyses of risk modulators
Although coefficient assignment for variables was not derived from PPMI and/or DeNoPa (but previously established13), univariate analysis in each model was used to confirm their association with the diagnosis of PD, such as via: the number and percentage of positive cases in each group; the odds ratio (OR); and an univariate AUC.
Constructing the PREDIGT questionnaire
For non-motor aspects of PD (i.e., constipation, hyposmia, depression, anxiety, and RBD), the PREDIGT Questionnaire was directly informed by the five questionnaires (MDS-UPDRS I; Scopa-AUT; PD NMSS; PD NMS; and PDQ-39) included in the DeNoPa clinical research forms. In PPMI, clinical research forms entailed two related questionnaires (MDS-UPDRS I; Scopa-AUT) but none related to olfaction. Thus, additional analyses with fewer shared variables were performed using both PPMI + FOUND and DeNoPa cohorts. The list of questionnaires analyzed are shown in Supplementary Table 2; those selected for model purposes are listed in Supplementary Table 3.
Calculating an individual’s PREDIGT Score
After reviewing available information in the two cohorts, 13 variables entered phase 1 of our study (Supplementary Table 1). These were transformed into a binary format (positive/negative) or ordinal format with multiple levels and then multiplied by the previously published coefficients, based on criteria shown in Supplementary Table 113. The PREDIGT Score of each participant was then calculated as per the formula: PR = (E + D + I) x G x T. In order to ensure all participants had scores >0 (that would increase with progression in age), age-associated minimal values for factors ‘E’, ‘D’, and ‘I’ were included, as previously described13 (see also details in Supplementary Table 1).
We evaluated the discrimination ability of the PREDIGT Score between PD patients and HC in a cross-sectional analysis. ROC curves and AUC values were calculated. Confidence intervals (CI) of reported AUC values were estimated using bootstrapping with 1000 replications. Reported sensitivity and specificity were relative to the optimal threshold identified using the Youden Index46. Score density plots were used to illustrate the degree of distinction between groups. Two versions of the PREDIGT Score were assessed (Model 1 and Model 2).
Model 1: PREDIGT Score
Model 2: 2-step PREDIGT Score
For population screening purposes, we created a two-step model using first self-reported variables (Step 1) and a single, objective test (Step 2). The screening-focused questionnaire was derived from the original PREDIGT Score model (Supplementary Table 3) and was assembled using exclusively self-reported data. The screening-oriented PREDIGT Score was then calculated using the same formula and coefficients as above. The Step 1 threshold, C1, was determined by the largest combined AUC that could be achieved. Objective markers assessed in Step 2 are listed in Supplementary Table 4; the Step 2 threshold, C2, was determined by the maximum of Youden Index.
Applications for DeNoPa data are reviewed by the study investigators (Drs. Brit Mollenhauer and Claudia Trenkwalder; see also at https://www.denopa.de/Home.29.0.html). Applications for PPMI (FOUND) data are reviewed by the PPMI steering committee (https://www.ppmi-info.org/access-data-specimens/download-data).
All analyses were performed using R version 3.6.0. Codes for data analysis is available via https://github.com/JuanLiOHRI/PREDIGT.
Collaborators Global Burden of Disease (GBDPsD). Global, regional, and national burden of Parkinson’s disease, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. 17, 939–953 (2018).
Dorsey, E. R. et al. Projected number of people with Parkinson's disease in the most populous nations, 2005 through 2030. Neurology 68, 384–386 (2007).
Espay, A. J. et al. Technology in Parkinson’s disease: Challenges and opportunities. Mov. Disord. 31, 1272–1282 (2016).
Berg, D. et al. The PRIPS study: screening battery for subjects at risk for Parkinson’s disease. Eur. J. Neurol. 20, 102–108 (2013).
Nalls, M. A. et al. Diagnosis of Parkinson’s disease on the basis of clinical and genetic classification: a population‐based modelling study. Lancet Neurol. 14, 1002–1009 (2015).
Schrag, A., Anastasiou, Z., Ambler, G., Noyce, A. & Walters, K. Predicting diagnosis of Parkinson’s disease: A risk algorithm based on primary care presentations. Mov. Disord. 34, 480–486 (2019).
Bestwick, J. P. et al. Improving estimation of Parkinson’s disease risk—the enhanced PREDICT-PD algorithm. npj Parkinsons Dis. 7, 33 (2021).
Berg, D. et al. MDS research criteria for prodromal Parkinson’s disease. Mov. Disord. 30, 1600–1611 (2015).
Heinzel, S. et al. Update of the MDS research criteria for prodromal Parkinson’s disease. Mov. Disord. 34, 1464–1470 (2019).
Mahlknecht, P. et al. Prodromal Parkinson’s disease as defined per MDS research criteria in the general elderly community. Mov. Disord. 31, 1405–1408 (2016).
Fereshtehnejad, S. M. et al. Validation of the MDS research criteria for prodromal Parkinson’s disease: Longitudinal assessment in a REM sleep behavior disorder (RBD) cohort. Mov. Disord. 32, 865–873 (2017).
Mirelman, A. et al. Application of the Movement Disorder Society prodromal criteria in healthy G2019S-LRRK2 carriers. Mov. Disord. 33, 966–973 (2018).
Schlossmacher, M. G. et al. Modelling idiopathic Parkinson disease as a complex illness can inform incidence rate in healthy adults: the PREDIGT Score. Eur. J. Neurosci. 45, 175–191 (2017).
Kalia, L. V. & Lang, A. E. Parkinson’s disease. Lancet 386, 896–912 (2015).
Mollenhauer, B. et al. Nonmotor and diagnostic findings in subjects with de novo Parkinson disease of the DeNoPa cohort. Neurology 81, 1226–1234 (2013).
Parkinson Progression. The Parkinson progression marker initiative (PPMI). Prog. Neurobiol. 95, 629–635 (2011).
2015 PPMI Annual Meeting (2015) FOUND & Fox Insight in PPMI. Online document: https://www.ppmi-info.org/wp-content/uploads/2015/06/08c_v1_Tanner_Arnado_FOUND.PPMI-FI.2015.ANN-.-MTG..pdf.
Ritz, B. & Rhodes, S. L. After half a century of research on smoking and PD, where do we go now?. Neurology 74, 870–871 (2010).
Postuma, R. B. et al. Risk and predictors of dementia and Parkinsonism in idiopathic REM sleep behaviour disorder: a multicentre study. Brain 142, 744–759 (2019).
Tokuda, T. et al. Decreased alpha-synuclein in cerebrospinal fluid of aged individuals and subjects with Parkinson’s disease. Biochem Biophys. Res Commun. 349, 162–166 (2006).
Mollenhauer, B. et al. Direct quantification of CSF alpha-synuclein by ELISA and first cross-sectional study in patients with neurodegeneration. Exp. Neurol. 213, 315–325 (2008).
Mollenhauer, B. et al. α-Synuclein and tau concentrations in cerebrospinal fluid of patients presenting with parkinsonism: a cohort study. Lancet Neurol. 10, 230–240 (2011).
Egeberg, A., Hansen, P. R., Gislason, G. H. & Thyssen, J. P. Exploring the association Between Rosacea and Parkinson disease: A Danish Nationwide Cohort Study. JAMA Neurol. 73, 529–534 (2016).
Hui, K. Y. et al. Functional variants in the LRRK2 gene confer shared effects on risk for Crohn’s disease and Parkinson’s disease. Sci. Transl. Med. 10, eaai7795 (2018).
Tsai, H.-H. et al. Hepatitis C virus infection as a risk factor for Parkinson disease: A nationwide cohort study. Neurology 86, 840–846 (2016).
Eidson, L. N. et al. Candidate inflammatory biomarkers display unique relationships with alpha-synuclein and correlate with measures of disease severity in subjects with Parkinson’s disease. J. Neuroinflammation 14, 164 (2017).
Shutinoski, B. et al. Lrrk2 alleles modulate inflammation during microbial infection of mice in a sex-dependent manner. Sci. Transl. Med. 11, 1–16 (2019).
Bellou, V., Belbasis, L., Tzoulaki, I., Evangelou, E. & Ioannidis, J. P. Environmental risk factors and Parkinson’s disease: An umbrella review of meta-analyses. Parkinsonism Relat. Disord. 23, 1–9 (2016).
Van Den Eeden, S. K. et al. Incidence of Parkinson’s disease: variation by age, gender, and race/ethnicity. Am. J. Epidemiol. 157, 1015–1022 (2003).
Groveman, B. R. et al. Rapid and ultra-sensitive quantitation of disease-associated α-synuclein seeds in brain and cerebrospinal fluid by αSyn RT-QuIC. Acta Neuropathol. 6, 7 (2018). Erratum in: Acta Neuropathol. Commun. 8, 180 (2020).
van Rumund, A. et al. α-Synuclein real-time quaking-induced conversion in the cerebrospinal fluid of uncertain cases of parkinsonism. Ann. Neurol. 85, 777–781 (2019).
Shahnawaz, M. et al. Discriminating α-synuclein strains in Parkinson’s disease and multiple system atrophy. Nature 578, 273–277 (2020).
Poewe, W. & Espay, A. J. Long duration response in Parkinson’s disease: levodopa revisited. Brain 143, 2332–2335 (2020).
Trinh, J. & Farrer, M. Advances in the genetics of Parkinson disease. Nat. Rev. Neurol. 9, 445–454 (2013).
Nalls, M. A. et al. Identification of novel risk loci, causal insights, and heritable risk for Parkinson’s disease: a meta-analysis of genome-wide association studies. Lancet Neurol. 18, 1091–1102 (2019).
Morley, J. F. et al. Optimizing olfactory testing for the diagnosis of Parkinson’s disease: item analysis of the University of Pennsylvania smell identification test. npj Parkinson’s Dis. 4, 2 (2018).
Heintz-Buschart, A. et al. The nasal and gut microbiome in Parkinson’s disease and idiopathic rapid eye movement sleep behavior disorder. Mov. Disord. 33, 88–98 (2018).
Manuel, D. G. et al. Measuring burden of unhealthy behaviours using a multivariable predictive approach: life expectancy lost in Canada attributable to smoking, alcohol, physical inactivity, and diet. PLoS Med. 13, e1002082 (2016).
Noyce, A. J., Lees, A. J. & Schrag, A. The prediagnostic phase of Parkinson’s disease. J. Neurol. Neurosurg. Psychiatry 87, 871–878 (2016).
Jennings, D. et al. Imaging prodromal Parkinson disease: the Parkinson Associated Risk Syndrome study. Neurology 83, 1739–1746 (2014).
Kotecha, A. M., Corrêa, A. D. C., Fisher, K. M. & Rushworth, J. V. Olfactory dysfunction as a global biomarker for sniffing out Alzheimer’s disease: A meta-analysis. Biosensors. 8, 41 (2018).
Fisher, S. et al. Development and validation of a predictive algorithm for risk of dementia in the community setting. J. Epidemiol. Community Health 75, 843–853 (2021).
Hughes, A. J., Daniel, S. E., Kilford, L. & Lees, A. J. Accuracy of clinical diagnosis of idiopathic Parkinson’s disease. A clinico-pathological study of 100 cases. J. Neurol. Neurosurg. Psychiatry 55, 181–184 (1992).
Hummel, T., Kobal, G., Gudziol, H. & Mackay-Sim, A. Normative data for the “Sniffin’ Sticks” including tests of odor identification, odor discrimination, and olfactory thresholds: an upgrade based on a group of more than 3000 subjects. Eur. Arch. Otorhinolaryngol. 264, 237–243 (2007).
Muirhead N., Benjamin E. & Saleh H. A. Is the University of Pennsylvania Smell Identification Test (UPSIT) valid for the UK population? Otorhinolaryngologist 6, 99–103 (2013).
Youden, W. J. Index for rating diagnostic tests. Cancer 3, 32–35 (1950).
Harrell, F. E. Jr. rms: Regression Modeling Strategies. R package version 5.1–4, https://cran.r-project.org/web/packages/rms/index.html (2019).
Wickham H. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. ISBN 978-3-319-24277-4, https://ggplot2.tidyverse.org (2016).
The authors acknowledge the commitment of study participants in the DeNoPa and PPMI cohorts and are grateful to all clinical research coordinators at their participating study sites. PPMI data used in the preparation of this article were obtained from the Parkinson’s Progression Markers Initiative (PPMI) database (www.ppmi-info.org/data). For up-to-date information on the study, visit www.ppmi-info.org. We thank Dr. Anna Naito for help in data acquisition and coordination. We thank Ms. Nathalie Lengacher for help in graphic design. This work was supported by funding from Parkinson Canada (to T.A.M., D.M., M.G.S; 2018; to J.L.; 2019-2021), Michael J. Fox Foundation for Parkinson’s Research (MJFF-006231; to T.A.M., D.M., M.G.S), Department of Medicine (T.M., T.R., D.M., M.G.S.), The Ottawa Hospital Foundation (Borealis Foundation to J.L.) and the Uttra & Sam Bhargava Family (M.G.S.), the Paracelsus-Elena-Klinik Kassel, Parkinson Fonds Deutschland, and the Deutsche Parkinson Vereinigung (B.M.; C.T.). PPMI, as a public-private partnership, is funded by the Michael J. Fox Foundation for Parkinson’s Research and its funding partners (a list of the full names of all PPMI funding partners can be found at: www.ppmi-info.org/fundingpartners). The funders had no role in the design and execution of the study; the collection, management, analysis, and interpretation of the data; the preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication. We are grateful for the ongoing support and feedback from people with lived experiences, such as through the board of the Parkinson’s Research Consortium Ottawa and members of Partners Investing in Parkinson’s Research, and to Drs. P. Wells and D. Lewis for their ongoing encouragement.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Li, J., Mestre, T.A., Mollenhauer, B. et al. Evaluation of the PREDIGT score’s performance in identifying newly diagnosed Parkinson’s patients without motor examination. npj Parkinsons Dis. 8, 94 (2022). https://doi.org/10.1038/s41531-022-00360-5