Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Remote smartphone monitoring of Parkinson’s disease and individual response to therapy

Abstract

Remote health assessments that gather real-world data (RWD) outside clinic settings require a clear understanding of appropriate methods for data collection, quality assessment, analysis and interpretation. Here we examine the performance and limitations of smartphones in collecting RWD in the remote mPower observational study of Parkinson’s disease (PD). Within the first 6 months of study commencement, 960 participants had enrolled and performed at least five self-administered active PD symptom assessments (speeded tapping, gait/balance, phonation or memory). Task performance, especially speeded tapping, was predictive of self-reported PD status (area under the receiver operating characteristic curve (AUC) = 0.8) and correlated with in-clinic evaluation of disease severity (r = 0.71; P < 1.8 × 10−6) when compared with motor Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). Although remote assessment requires careful consideration for accurate interpretation of RWD, our results support the use of smartphones and wearables in objective and personalized disease assessments.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: Data collection, processing and analysis of active performance assessments.
Fig. 2: Personalized fluctuations in performance across assessment and association with on/off motor fluctuations and diurnal variations.
Fig. 3: Association between mPower active assessments collected outside clinic and disease severity as measured in-clinic in the objectivePD cohort.

Data availability

The raw data from 67% of participants who have chosen to share their data broadly with all qualified researchers is available at https://doi.org/10.7303/syn4993293. Features, intermediate results and trained models for these participants are also available in Synapse (https://doi.org/10.7303/syn23277418). Access to data requires users to have their Synapse account validated, submit a data use statement and agree to terms of use. To aid in reproducibility and provide all final and intermediate results to the research community, we have redone the analysis presented here using the broadly shared data.

Code availability

The code, a docker container with all installed packages and a snakemake script that reproduces all of the figure and analysis, is available in GitHub: https://github.com/Sage-Bionetworks/mPowerRerun.

References

  1. Sherman, R. E. et al. Real-world evidence—what is it and what can it tell us? N. Engl. J. Med. 375, 2293–2297 (2016).

    Article  PubMed  Google Scholar 

  2. Steinhubl, S. R. et al. Effect of a home-based wearable continuous ECG monitoring patch on detection of undiagnosed atrial fibrillation: the mSToPS randomized clinical trial. JAMA 320, 146–155 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  3. Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Effectiveness of continuous glucose monitoring in a clinical care environment: evidence from the Juvenile Diabetes Research Foundation continuous glucose monitoring (JDRF-CGM) trial. Diabetes Care 33, 17–22 (2010).

  4. Anguera, J. A., Jordan, J. T., Castaneda, D., Gazzaley, A. & Areán, P. A. Conducting a fully mobile and randomised clinical trial for depression: access, engagement and expense. BMJ Innov. 2, 14–21 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Quer, G. et al. Home monitoring of blood pressure: short-term changes during serial measurements for 56398 subjects. IEEE J. Biomed. Health Inform. 22, 1691–1698 (2018).

    Article  PubMed  Google Scholar 

  6. Chaibub Neto, E. et al. Learning Disease vs Participant Signatures: a permutation test approach to detect identity confounding in machine learning diagnostic applications. Preprint at https://arxiv.org/abs/1712.03120 (2017).

  7. Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C. & Kording, K. P. The need to approximate the use-case in clinical machine learning. Gigascience 6, 1–9 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Dorsey, E. R. et al. The use of smartphones for health research. Acad. Med. 92, 157–160 (2017).

    Article  PubMed  Google Scholar 

  9. Pratap, A. et al. Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. NPJ Digit. Med. 3, 21 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Arora, S. et al. Detecting and monitoring the symptoms of Parkinson’s disease using smartphones: a pilot study. Parkinsonism Relat. Disord. 21, 650–653 (2015).

    Article  CAS  PubMed  Google Scholar 

  11. Espay, A. J. et al. Technology in Parkinson’s disease: challenges and opportunities. Mov. Disord. 31, 1272–1282 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  12. Ellis, R. J. et al. A validated smartphone-based assessment of gait and gait variability in Parkinson’s disease. PLoS ONE 10, e0141694 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Kostikis, N., Hristu-Varsakelis, D., Arnaoutoglou, M. & Kotsavasiloglou, C. A smartphone-based tool for assessing Parkinsonian hand tremor. IEEE J. Biomed. Health Inform. 19, 1835–1842 (2015).

    Article  CAS  PubMed  Google Scholar 

  14. Goetz, C. G. et al. Movement Disorder Society–sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results. Mov. Disord. 23, 2129–2170 (2008).

    Article  PubMed  Google Scholar 

  15. Hauser, R. A. et al. A home diary to assess functional status in patients with Parkinson’s disease with motor fluctuations and dyskinesia. Clin. Neuropharmacol. 23, 75–81 (2000).

    Article  CAS  PubMed  Google Scholar 

  16. Heldman, D. A. et al. The modified bradykinesia rating scale for Parkinson’s disease: reliability and comparison with kinematic measures. Mov. Disord. 26, 1859–1863 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  17. Jeon, H. et al. Automatic classification of tremor severity in Parkinson’s disease using a wearable device. Sensors 17, 2067 (2017).

    Article  PubMed Central  Google Scholar 

  18. Stamate, C. et al. Deep learning Parkinson’s from smartphone data. in Proc. 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom) 31–40 (IEEE, 2017).

  19. Liddle, J. et al. Measuring the lifespace of people with Parkinson’s disease using smartphones: proof of principle. JMIR Mhealth Uhealth 2, e13 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Ginis, P. et al. Feasibility and effects of home-based smartphone-delivered automated feedback training for gait in people with Parkinson’s disease: a pilot randomized controlled trial. Parkinsonism Relat. Disord. 22, 28–34 (2016).

    Article  PubMed  Google Scholar 

  21. Stamate, C. et al. The cloudUPDRS app: a medical device for the clinical assessment of Parkinson’s Disease. Pervasive Mob. Comput. 43, 146–166 (2018).

    Article  Google Scholar 

  22. Lipsmeier, F. et al. Evaluation of smartphone-based testing to generate exploratory outcome measures in a phase 1 Parkinson’s disease clinical trial. Mov. Disord. 33, 1287–1297 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  23. Zhan, A. et al. Using smartphones and machine learning to quantify Parkinson disease severity: The mobile Parkinson disease score. JAMA Neurol. 75, 876–880 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Bot, B. M. et al. The mPower study, Parkinson disease mobile data collected using ResearchKit. Sci. Data 3, 160011 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Chaibub Neto, E. et al. On the analysis of personalized medication response and classification of case vs control patients in mobile health studies: the mPower case study. Preprint at https://arxiv.org/abs/1706.09574 (2017).

  26. Chaibub Neto, E., Tummalacherla, M., Mangravite, L. & Omberg, L. Causality-based tests to detect the influence of confounders on mobile health diagnostic applications: a comparison with restricted permutations. in Machine Learning for Health (ML4H) Workshop at NeurIPS 2019. https://arxiv.org/abs/1911.05139 (2019).

  27. Little, M. A. et al. Using and understanding cross-validation strategies. Perspectives on Saeb et al. Gigascience 6, 1–6 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Neto, E. C. et al. Detecting the impact of subject characteristics on machine learning-based diagnostic applications. NPJ Digit. Med. 2, 1–6 (2019).

    Google Scholar 

  29. Sakar, B. E. et al. Collection and analysis of a Parkinson speech dataset with multiple types of sound recordings. IEEE J. Biomed. Health Inform. 17, 828–834 (2013).

    Article  PubMed  Google Scholar 

  30. Mishra, S. R. et al. Supporting coping with Parkinson’s disease through self-tracking. in Proc. 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19) (ACM) 1–16 (2019).

  31. Newey, W. K. & West, K. D. A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55, 703 (1987).

    Article  Google Scholar 

  32. Newey, W. K. & West, K. D. Automatic lag selection in covariance matrix estimation. Rev. Econ. Stud. 61, 631–653 (1994).

    Article  Google Scholar 

  33. Grömping, U. Estimators of relative importance in linear regression based on variance decomposition. Am. Stat. 61, 139–147 (2007).

    Article  Google Scholar 

  34. Thenganatt, M. A. & Jankovic, J. Parkinson disease subtypes. JAMA Neurol. 71, 499 (2014).

    Article  PubMed  Google Scholar 

  35. Khazaal, Y. et al. Does self-selection affect samples’ representativeness in online surveys? An investigation in online video game research. J. Med. Internet Res. 16, e164 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Bethlehem, J. Selection bias in web surveys. Int. Stat. Rev. 78, 161–188 (2010).

    Article  Google Scholar 

  37. Badawy, R. et al. Automated quality control for sensor based symptom measurement performed outside the lab. Sensors 18, 1215 (2018).

    Article  PubMed Central  Google Scholar 

  38. Chaibub Neto, E. et al. Towards personalized causal inference of medication response in mobile health: an instrumental variable approach for randomized trials with imperfect compliance. Preprint at https://arxiv.org/abs/1604.01055 (2016).

  39. Barnett, I., Torous, J., Staples, P., Keshavan, M. & Onnela, J.-P. Beyond smartphones and sensors: choosing appropriate statistical methods for the analysis of longitudinal data. J. Am. Med. Inform. Assoc. 25, 1669–1674 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Marbach, D. et al. Wisdom of crowds for robust gene network inference. Nat. Methods 9, 796–804 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Tavares, A. L. T. et al. Quantitative measurements of alternating finger tapping in Parkinson’s disease correlate with UPDRS motor disability and reveal the improvement in fine motor control from medication and deep brain stimulation. Mov. Disord. 20, 1286–1298 (2005).

    Article  Google Scholar 

  42. Tsanas, A., Little, M. A., McSharry, P. E. & Ramig, L. O. Nonlinear speech analysis algorithms mapped to a standard metric achieve clinically useful quantification of average Parkinson’s disease symptom severity. J. R. Soc. Interface 8, 842–855 (2011).

    Article  PubMed  Google Scholar 

  43. Sejdic, E., Lowry, K. A., Bellanca, J., Redfern, M. S. & Brach, J. S. A comprehensive assessment of gait accelerometry signals in time, frequency and time-frequency domains. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 603–612 (2014).

    Article  PubMed  Google Scholar 

  44. Sieberts, S. K. et al. Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge. NPJ Digit. Med. https://doi.org/10.1038/s41746-021-00414-7 (2021).

  45. Doerr, M. et al. Formative evaluation of participant experience with mobile econsent in the app-mediated Parkinson mPower study: a mixed methods study. JMIR Mhealth Uhealth 5, e14 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Gill, D. J., Freshman, A., Blender, J. A. & Ravina, B. The Montreal cognitive assessment as a screening tool for cognitive impairment in Parkinson’s disease. Mov. Disord. 23, 1043–1046 (2008).

    Article  PubMed  Google Scholar 

  47. Weintraub, D., Oehlberg, K. A., Katz, I. R. & Stern, M. B. Test characteristics of the 15-item geriatric depression scale and Hamilton depression rating scale in Parkinson disease. Am. J. Geriatr. Psychiatry 14, 169–175 (2006).

    Article  PubMed  PubMed Central  Google Scholar 

  48. Schwab, R.S. & England, A.C. Projection technique for evaluating surgery in Parkinson’s disease. in Third Symposium on Parkinson’s Disease (eds Gillingham, F. J. & Danoldson, I. M. L.) 152–157 (E & S. Livingston, 1969).

  49. McRae, C., Diem, G., Vo, A., O’Brien, C. & Seeberger, L. Schwab & England: standardization of administration. Mov. Disord. 15, 335–336 (2000).

    Article  CAS  PubMed  Google Scholar 

  50. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/ (2014).

  51. Taylor, A. L. T. et al. Quantitative measurements of alternating finger tapping in Parkinsons disease correlate with UPDRS motor disability and reveal the improvement in fine motor control from medication and deep brain stimulation. Mov. Disord. 20, 1286–1298 (2005).

    Article  Google Scholar 

  52. Ho, D., Imai, K., King, G. & Stuart, E. A. MatchIt: non-parametric preprocessing for parametric causal inference. J. Stat. Softw. 42, 1–28 (2011).

    Article  Google Scholar 

  53. Rousseeuw, P. J. & Croux, C. Alternatives to the median absolute deviation. J. Am. Stat. Assoc. 88, 1273–1283 (1993).

    Article  Google Scholar 

  54. Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).

    Article  Google Scholar 

  55. Liaw, A. & Wiener, M. Classification and regression by randomForest. R News 2, 18–22 (2002).

    Google Scholar 

  56. Hastie, T, Tibshirani, R & Friedman, J. The Elements of Statistical Learning (Springer, 2001).

  57. Friedman, J., Hastie, T. & Tibshirani, R. Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33, 1–22 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  58. Pearl, J. Causality: Models, Reasoning, and Inference (Cambridge University Press, 2009).

  59. Pearl, J. & Mackenzie, D. The Book of Why: the New Science of Cause and Effect (Basic Books, 2018). .

  60. Szekely, G., Rizzo, M. L. & Bakirov, N. K. Measuring and testing dependence by correlation of distances. Ann. Stat. 35, 2769–2794 (2007).

    Article  Google Scholar 

  61. Szekely, G. & Rizzo, M. L. Partial distance correlation with methods for dissimilarities. Ann. Stat. 42, 2382–2412 (2014).

    Article  Google Scholar 

  62. Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C. & Kording, K. P. The need to approximate the use-case in clinical machine learning. GigaScience 6, 1–9 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  63. Chaibub Neto, E. et al. Detecting the impact of subject characteristics on machine learning-based diagnostic applications. NPJ Digit. Med. 2, 99 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  64. Sarkar, B. E. et al. Collection and analysis of a Parkinson speech dataset with multiple types of sound recordings. IEEE J. Biomed. Health Inform. 17, 828–834 (2013).

    Article  Google Scholar 

  65. Bamber, D. The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J. Math. Psychol. 12, 387415 (1975).

    Article  Google Scholar 

  66. Mason, S. L. & Graham, N. E. Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: statistical significance and interpretation. Q. J. R. Meteorolog. Soc. 128, 2145–2166 (2002).

    Article  Google Scholar 

  67. Newey, W. K. & West, K. D. A simple, positive-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55, 703708 (1987).

    Article  Google Scholar 

  68. Zeileis, A. Econometric computing with HC and HAC covariance matrix estimation. J. Stat. Softw. 10, 1–17 (2004).

    Google Scholar 

  69. Box, G., Jenkins, G. M. & Reinsel, G. C. Time Series Analysis: Forecasting and Control 3rd edn (Prentice-Hall, 1994).

  70. Hyndman, R. J. & Khandakar, Y. Automatic time series forecasting: the forecast package for R. J. Stat. Softw. 26, 1–22 (2008).

    Google Scholar 

  71. Datta, D. D. & Du W. Nonparametric HAC estimation for time series data with missing observations. International Finance Discussion Papers. The Federal Reserve Board (2012).

  72. Rho, S. H. & Vogelsang, T. J. Heteroskedasticity autocorrelation robust inference in time series regressions with missing data. Econometric Theory 35, 601–629 (2019).

    Article  Google Scholar 

  73. McGregor, J. R. & Babb, J. C. Serially correlated differences in the paired comparison of time series. Biometrika 76, 735–739 (1989).

    Article  Google Scholar 

  74. Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57, 289–300 (1995).

    Google Scholar 

  75. Lindeman, R. H., Merenda, P. F., and Gold, R. Z. Introduction to Bivariate and Multivariate Analysis (Scott, Foresman, 1980).

  76. Kruskal, W. Relative importance by averaging over orderings. Am. Stat. 41, 6–10 (1987).

    Google Scholar 

  77. Gromping, U. Relative importance for linear regression in R: the package relaimpo. J. Stat. Softw. 17, 1–27 (2007).

Download references

Acknowledgements

This work was funded through a grant from the Robert Wood Johnson Foundation. These data were contributed by users of the Parkinson mPower mobile application as part of the mPower study developed by Sage Bionetworks and described in Synapse (https://doi.org/10.7303/syn4993293).

Author information

Authors and Affiliations

Authors

Contributions

L.O., E.C.N. and L.M.M. wrote the paper. L.O. oversaw the analytical and feature extraction activities. L.M.M. was the principal investigator on the study. E.C.N. developed analytical methods. E.C.N. performed the analyses of the mPower data, assisted by T.M.P., A.P. and L.O. A.T. independently reproduced analyses of the mPower data. T.M.P., A.P. and E.C.N. developed features extraction pipelines and figures. B.M.B. curated data and consulted on analyses. A.K. helped design in-app language, look and logic. M.R.K. led the team that designed and implemented the mPower app, and contributed to study design and data capture methodology. E.R.D. designed and oversaw the ObjectivePD validation study with assistance from M.E. and R.S. C.S. and J.W. led the team that designed and implemented the governance for informed consent and data sharing. A.D.T. conceived the study and helped design the app. J.A. performed in-clinic data collection. B.R.B., S.M.G., K.K., M.A.L., C.T. and C.M.T. served as scientific advisors on the study. All authors assisted with revisions of the paper.

Corresponding authors

Correspondence to Larsson Omberg, Elias Chaibub Neto or Lara M. Mangravite.

Ethics declarations

Competing interests

B.R.B. currently serves as Editor-in-Chief for the Journal of Parkinson’s Disease, serves on the editorial boards of Practical Neurology and Digital Biomarkers, has received honoraria from serving on the scientific advisory board for Zambon, Biogen, UCB and Walk with Path, has received fees for speaking at conferences from AbbVie, Zambon, Roche, GE Healthcare and Bial, and has received research support from the Netherlands Organization for Scientific Research, the Michael J. Fox Foundation, UCB, Abbvie, Zambon, the Stichting Parkinson Fonds, the Hersenstichting Nederland, the Parkinson’s Foundation, Verily Life Sciences, Horizon 2020, the Topsector Life Sciences and Health and the Parkinson Vereniging. E.R.D. has received honoraria for speaking at American Academy of Neurology courses, American Neurological Association and University of Michigan; received compensation for consulting services from 23andMe, Abbott, Abbvie, American Well, Biogen, BrainNeuroBio, Clintrex, Curasen Therapeutics, DeciBio, Denali Therapeutics, GlaxoSmithKline, Grand Rounds, Karger, Lundbeck, MC10, MedAvante, Medical-legal services, Mednick Associates, National Institute of Neurological Disorders and Stroke, Olson Research Group, Optio, Origent Data Sciences, Inc., Otsuka, Prilenia, Putnam Associates, Roche, Sanofi, Shire, Spark, Sunovion Pharma, Teva, Theravance, UCB and Voyager Therapeutics; research support from Abbvie, Acadia Pharmaceuticals, AMC Health, Biosensics, Burroughs Wellcome Fund, Davis Phinney Foundation, Duke University, Food and Drug Administration, GlaxoSmithKline, Greater Rochester Health Foundation, Huntington Study Group, Michael J. Fox Foundation, the NIH/NINDS, NSF, Nuredis Pharmaceuticals, Patient-Centered Outcomes Research Institute, Pfizer, Prana Biotechnology, Raptor Pharmaceuticals, Roche, Safra Foundation, Teva Pharmaceuticals and University of California Irvine; editorial services for Karger Publications; and ownership interests with Grand Rounds (second opinion service). C.M.T. reports grants from Sage Biometrics, during the conducting of the study; grants from Parkinson Foundation, grants from Gateway LLC, grants from Roche/Genentech, grants from Parkinson Study Group, personal fees from Biogen Idec, personal fees from Acorda, personal fees from Adamas Therapeutics, personal fees from Amneal, personal fees from CNS Ratings, personal fees from Grey Matter LLC, grants from Michael J. Fox Foundation, grants from NIH/NIA, grants from NIH/NINDS, grants from VA Merit, grants from Department of Defense, personal fees from Northwestern University, personal fees from Partners, Harvard University, nonfinancial support from Medtronic, Inc., nonfinancial support from Acadia, nonfinancial support from Boston Scientific, nonfinancial support from Neurocrine, nonfinancial support from Acadia, grants from Biogen Idec research support, nonfinancial support from Biogen Idec, personal fees from Guidemark Health, personal fees from Acadia, personal fees from Neurocrine, personal fees from Lundbeck, personal fees from Cadent, nonfinancial support from Neurocrine and grants from Roche Genentech, outside the submitted work.

Additional information

Peer review information Nature Biotechnology thanks the anonymous reviewers for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–31 and Note 1.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Omberg, L., Chaibub Neto, E., Perumal, T.M. et al. Remote smartphone monitoring of Parkinson’s disease and individual response to therapy. Nat Biotechnol 40, 480–487 (2022). https://doi.org/10.1038/s41587-021-00974-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41587-021-00974-9

This article is cited by

Search

Quick links

Nature Briefing: Translational Research

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Get what matters in translational research, free to your inbox weekly. Sign up for Nature Briefing: Translational Research