Alzheimer’s Disease (AD) represents a major and rapidly growing burden to the healthcare ecosystem. A growing body of evidence indicates that cognitive, behavioral, sensory, and motor changes may precede clinical manifestations of AD by several years. Existing tests designed to diagnose neurodegenerative diseases, while well-validated, are often less effective in detecting deviations from normal cognitive decline trajectory in the earliest stages of the disease. In the quest for gold standards for AD assessment, there is a growing interest in the identification of readily accessible digital biomarkers, which harness advances in consumer grade mobile and wearable technologies. Topics examined include a review of existing early clinical manifestations of AD and a path to the respective sensor and mobile/wearable device usage to acquire domain-centric data towards objective, high frequency and passive digital phenotyping.
Alzheimer’s Disease (AD) represents a major and rapidly growing burden to the healthcare ecosystem. In the USA alone, some 5 million people suffer from the disease that costs the managed healthcare system in excess of $250 billion. Currently the sixth leading cause of death, AD prevalence has increased by 89% since 2000, underscoring the need for interventive and preventative measures. Despite enormous capital investments, drug development has been problematic. It is generally accepted that the likelihood of reversing anatomic and physiologic changes (e.g., neuronal death) decreases dramatically as the disease advances, placing increased attention on early cohort discovery and patient stratification for any future clinical studies. Accordingly, there is an acute need to detect the disease at prodromal stages. In this quest for monitoring biomarkers for AD assessment, there is growing interest in the identification of readily accessible digital biomarkers, which leverage widely available mobile and wearable technologies, and it is these that are the subject of this review article.
A growing body of evidence indicates that cognitive, sensory and motor changes may precede clinical manifestations of AD by several years.1 In particular, sensory and motor (non-cognitive) changes can help detect a neurological or neurodegenerative disease 10 or 15 years prior their effective diagnosis. This said, existing validated neuropsychological/cognitive tests designed to diagnose neurodegenerative diseases are often less effective in detecting deviations from normal cognitive trajectory in the earliest stages of the disease. Furthermore, cognitive tests can suffer from intrinsic cultural bias, take a relatively long time to administer, provide only episodic information, show “practice effect” or “ceiling effect,” and are rater dependent.2 Explorations into the inclusion of genetic testing, structural MRI imaging and PET molecular imaging of beta-amyloid and tau protein promise earlier detection of disease, though these tests are currently limited to research applications due to their cost and invasive nature. These limitations preclude repeated and frequent use to test an individual and specifically in the early pre-symptomatic stage.
Mobile and wearable digital consumer technology has the potential to overcome these limitations, and their application in AD detection has become an area of increased interest.
Mobile and wearable device- derived data
Mobile and wearable technologies (such as smart phones, tablets, smart watches, and rings, smart suits) present a unique opportunity to massively detect neurodegenerative diseases in a timely and economical fashion due to:
a) the widespread usage of such technologies
b) the immediate access of information due to the inherent connectivity
c) the increasing sensitivity and plurality of onboard sensors
d) the nature of these sensors that are uniquely equipped to study such physical and cognitive abilities or symptoms
e) the extremely low burden on the healthcare system, since these devices are increasingly in use by large segments of the population.
Onboard sensors at the heart of these systems are able to provide metrics by means of active (prompted) or passive (unnoticed) measurements, offering considerable flexibility in approach.
Active data collection occurs when a user is prompted to perform a measurement and/or enter a metric value, e.g., a digital e-assessment cognitive test that probes memory on a tablet to detect AD,3 or a prompted voice test that probes vocal cord tremor to detect PD.4 These measurements are usually targeted at addressing specific metrics that have previously been correlated with the disease.
Passive data collection occurs when metric values are acquired unbeknownst to the user, e.g, a smartwatch-based step counter that continuously estimates step symmetry and length or a smart ring-based continuous heart rate monitor that picks up heart rate variability (HRV). As such, daily interaction with mobiles/wearables can result in a rich, high-frequency longitudinal data set that can be mined for signatures of a disease—while using users as their own control. Passive data collection has several advantages, including,
high frequency or even continuous data acquisition,
objectivity (not influenced by user perspective and learning effects),
low patient burden, which can lead to higher adherence.
On the other hand, passive data collection may
be limited to particular metrics that can be collected non-actively,
be expensive computationally and storage wise,
requires complex analysis tools to extract useful information.
These issues notwithstanding, given the decline in cognitive function and memory experienced by AD patients, passive data collection provides a logical approach for developing methods for disease forecasting, detection and monitoring. An enormous opportunity is presented for technology developers and healthcare professionals to ideate on new clinical studies which can provide insights to both disease detection and symptom assessment.
This review article considers disease-relevant aspects of sensor and device design, data collection modalities, and a path to clinical grade digital phenotyping. A non-exhaustive summary of available sensors or digital senses on each wearable/mobile device is presented in Fig. 1.
Disease specific metrics and sensors
Sensor signals can individually or collectively provide metrics that can determine aspects of a domain that is affected by a particular disease or condition (Table 1)
Movement—gross motor function. Sensors: IMU, geopositioning
A majority of AD patients exhibit pyramidal and extrapyramidal motor impairments starting at an early stage of the disease, which precede signs of cognitive impairment by at least a decade.5
Gait speed, stride length and gait symmetry are statistically reduced and gait speed variability is increased.6 This can be measured actively, by performing a fixed distance or duration walking test, or passively, by monitoring a subject using fixed7 or portable equipment.8 Buracchio and colleagues9 found that there is an inflection point in longitudinal observations of gait speed 12.1 years before clinical diagnosis of MCI (Mild Cognitive Impairment) where the annual decline rate in gait speed goes from −0.005 m/s/y to a dramatic −.023 m/s/y. It has also been shown that central nervous system impairment is related to stance time variability, whereas sensory impairment is related to step length variability.10
Although gait metrics alone provide limited specificity for AD detection, their value could come by incorporation into a composite scoring system. IMU sensors on smart phones, watches, rings and patches can estimate such metrics with good accuracy (step count: −6.7 to 6.2% for smartphone apps; Stride length: <5% for median values).11 In addition, contact pressure sensors (sock, shoe) can provide even higher insight on gait characteristics such as stance/swing ratio (pressure correlation > 95%).12 Further accuracy improvements can be achieved by fusing geopositioning information. Longitudinal monitoring of these metrics can help create composite disease predictors.
Movement—fine motor control. Sensors: touch screen, keyboard & stylus
Fine motor control and more particularly finger tapping speed and tracing accuracy have long been probed as potential early signs of AD. The finger tapping test is an active test where the subject is asked to tap beat a button as fast and as regularly as they can for a period of time; the total number of taps is recorded. While tapping speed normally decreases with age at a rate of −0.03 taps/y, the speed after the inflection point (2.66 y prior to clinical manifestation of the disease) dramatically decreased to a rate of −0.15 taps/y.9 Rabinowitz and colleagues13 showed that the contact time in a tapping test for subjects having an MMSE < 23 (a cognitive test used to evaluate AD: 20−24 is considered mild dementia, >24 is considered Normal) was increased by 38%, suggesting a much slower reaction speed. Tapping tests have already been implemented in smart phone applications with good patient adherence14 with a primary focus on Parkinson’s disease monitoring. Moreover, finger tapping speed has been correlated (r = 0.77) to inter-keystroke interval (typing speed),15 hence the potential for high frequency data collection from daily computer/tablet keyboard use. More recently, it has been shown16 that the text keystrokes per minute (excluding non-text keystrokes) as well as the number of pauses while typing, can discriminate between cognitive impairment (128.48 ± 35.03 keystrokes per minute) and healthy controls (63.65 ± 32.64 keystrokes per minute). Fine motor control can also be probed by looking at the accuracy of a digital pen motion, as in a tracing test, administered with a digital pen and tablet; the standard delineation (RMS distance) from the actual shape is calculated17 and was found to correlate with visuomotor performance and age.
Since one of the main means of interaction with mobile equipment is typing or drawing using a stylus, probing for typing speed and pauses while typing as well as pen trajectories present an excellent opportunity to longitudinally evaluate early signs of AD.
Speech and language. Sensor: microphone
Many aspects of language including grammatical and informational content as well as speech characteristics show deficits with increased AD progression. Using a combination of metrics such as periodic and aperiodic segment lengths, vocal reaction time, relative length (question/response), the amount of insertions/deletions and other irregularity traits, Konig and colleagues18 were able to classify between healthy controls, MCI and AD patients with an success rate of up to 87%. Fraser showed that the use of semantic, syntactic and acoustic voice features in a short picture-describing narration test can increase specificity of the disease and its stage.19 Even the simple metric of quantifying between-utterance pauses was shown to correlate with episodic memory that is associated to AD.20 In another simplified metrics study,21 the proportion of spoken words in a discussion (user vs interlocutor) was shown to positively correlate with transitions from normal cognition to MCI in the pre-symptomatic phase.
Conversations over mobile phones or between user and digital assistants are an excellent source of dense speech input. The available automatic speech recognition technologies (for example Google Assistant, Apple Siri, Microsoft Cortana, Amazon Alexa) claim high accuracy and can be used for transcription of subject discussions to be further analyzed. Another good source of language metrics is the keyboard-entered text on a mobile phone or tablet. As such, the syntactic and semantic analysis of spoken or written language can reveal early signs of the disease in a longitudinal, passive manner.
Occulomotor. Sensors: camera, light sensor
Eye movements and pupillary reflex have been used for several decades in neurological disease research. Careful examination of both allows to probe the medial temporal lobe memory system,22 the cholinergic neuronal pathways,23 the progressive neuropathological changes within the newcortex24,25 and the brain dopamine activity.26
The Visual Paired Comparison active test is administered by presenting on a screen a series of image pairs to a subject; these pairs include images that have previously been shown to the user.27 Healthy control eyes consistently perform more fixations on the novel image, whereas pre-AD subjects do not.22 Given the amount of new information we all receive from our tablets, a passive test that measures the fixation time on each new or old graphic presented to a user could be devised in order to quantify the extent of neuronal loss in the medial temporal lobe.
Pupillary constriction and dilation in response to light intensity changes is an efficient way to evaluate the central cholinergic dysfunction and consists a balance of forces exerted by the iris sphincter and dilator muscles. In an active test developed previously23,28 where a light flashes while the subject eyes are recorded using a high speed camera, it was shown that AD patients had significantly lower pupil constriction velocity and acceleration. Similarly, it was shown that pupillary reflex caused by abrupt changes in the illumination in a room were significantly different between patients with AD and controls.29 Today’s phone and tablet cameras have enough resolution to capture pupil diameter at high frame rate, thus providing the potential for a high frequency, pupillary reflex passive data collection.
Eye movements in reading
Reading is a complex process that involves optical sensory function, cognitive processing of incoming information and occulomotor functions. Using standard text and a high speed eye tracker, researchers showed that patients with early AD exhibited reduced number of words per fixation, an increase in the total number and duration of fixations and and increase in the number of words skipped.30,42 Given that reading is one of the basic functions performed on tablets, an on board high frequency eye movement data collection system while text is presented can provide insight to the stage of the disease. In addition, AD subjects show increased latency decreased eye movement velocity, and also have trouble fixating on a target.31 In a recent study, AD patients exhibited longer maximum fixation times compared to controls (2908 vs 1951 ms, respectively) as well as a higher number of large intrusive saccades (2.5 vs 0.7 respectively per test).32
Eye Blink Rate has also been examined26 to be a potential biomarker of Mild Cognitive Impairment, with MCI participants having a higher blink rate per minute than healthy controls (27.60 ± 15.09 versus 20.24 ± 13.24). Given the time spent in front of a tablet or phone screen, longitudinal blink rate changes can be picked up by the face camera and can be used in the digital biomarker arsenal for early disease detection.
Effective eye tracking is possible real time, given the high resolution of tablet and phone cameras. Numerous programs are currently active in developing video-based eye tracking solutions that can be incorporated in future operating systems of mobile devices, probing continuously for metrics that relate to AD.
Autonomic nervous system function. Sensors: PPG, ECG, ballistocardiography
A key hallmark of AD is the disruption of the cholinergic system of the brain; the resulting acetylcholine deficiencies are tied to many of the higher order cognitive symptoms such as memory loss and attention deficits21 and are related to severity of dementia.28 This also results in downstream physical symptoms through the disruption of the Autonomic Nervous System (ANS), whose parasympathetic system is heavily dependent on acetylcholine.33 Since both cholinergic34 and autonomic brainstem nuclei35 are amongst the earliest areas of the brain affected by AD-related tau aggregation, preceding cognitive symptoms by years, ANS disruptions represent a compelling opportunity to identify AD early on.
An important marker of ANS balance is Heart Rate Variability (HRV), a measure of the time intervals between heartbeats, resulting from the dual modulation of the heart by the sympathetic and parasympathetic systems. Due to the bidirectional vagal innervation between the heart and the brain, HRV has also been put forward as an index of cognitive function and stress.33 In healthy adults, lower Heart Rate Variability (and thus suppressed parasympathetic activity) was correlated with cognitive function,36 attention and working memory,33 mental stress37 and social cognition.38 In AD and MCI populations, where Parasympathetic function is suppressed due to damage to the cholinergic systems, HRV has been found to be lower than healthy controls as well as being negatively correlated to the level of cognitive function.39 Given the ties between HRV and cognition, the early disruption of the cholinergic and parasympathetic system as well as its progressive decline in AD, HRV serves as a compelling marker for AD progression.
The advent and increased prevalence of fitness trackers and smart watches equipped with photoplethysmography (PPG) capability set the foundation for widespread passive and continuous monitoring of users’ heart rate and heart rate variability.40 Although there are varying levels of validity between different devices when compared with ECG,41 studies have shown that in certain conditions (e.g., resting still) and for certain devices, HRV measurements were accurate enough to be used as detection tools.42,43 Proof-of-concept studies have used the data from devices with significant consumer uptake such as Apple Watche (Apple Inc, Cupertino, CA) and Android Wear (Google Inc., Mountain View, CA) to detect conditions passively such as atrial fibrillation with moderate amounts of accuracy.44,45 Smart rings equipped with PPG, such as the Oura ring (Oura Health Ltd, Oulu, Finland), have shown high reliability in measuring HR and HRV when compared to Electrocardiography (ECG).46 Consumer Ballistocardiography devices, in the form of sensing pads placed on the mattress, have also been shown to be capable of passively measuring HRV47 and detecting arrhythmias48 based on HRV signals. Another way to probe for the sympathetic nervous system condition (mainly arousal periods) is to measure skin resistance that varies with the state of sweat glands using a Galvanic Skin Response (GSR) sensor.49 A series of wrist-worn devices are equipped with such sensors, Empatica’s E4 (Empatica, Milan, Italy) and Verily’s Study Watch (Verily, South San Francisco, CA) have a validated track record of determining stress/anxiety during activities. Other emerging heart monitor modalities include the use of wrist or finger derived ECG from a wristband or a smartphone (KardiaBand and KardiaMobile, Alivecor, Mountain View, CA; Apple Watch, Cuppertino, CA), demonstrated to detect atrial fibrillation and tachycardia.50,51
Sleep patterns. Sensors: PPG, microphone, IMU, ballistocardiography
A commonly reported feature of AD has been circadian rhythm disruption in the form of sundowning or sleep fragmentation. Sleep studies of AD populations have confirmed these phenomena with sleep lab research indicating that patients with AD experienced more night-time awakenings, less time in REM sleep and lower sleep efficiency.52,53 Furthermore, the level of sleep disruption appeared to track with the level of cognitive deficit.53,54 Sleep disruption is corroborated by the biological changes in AD as the disease attacks the basal forebrain structures of the cholinergic55 and raphe nuclei of the serotonergic systems34 that contribute to sleep. Components of these systems are among the earliest affected brain areas and see changes in the prodromal stages of AD before cognitive decline.56,57
Indeed, there appears to be a bidirectional relationship between sleep quality and AD as studies have shown that sleep fragmentation contributes to developing Tau and AB pathologies, increasing the risk of developing AD.58 Accordingly, sleep quality could serve as an important indication of the early stages of AD.
App based and most wrist worn sleep monitors with a few exceptions, were shown to offer limited reliability at determining sleep stages when compared to Polysomnography (PSG).59 At the forefront, the Pulse-On (PulseOn Oy, Espoo, Finland) wearable device and the Oura Ring, have shown high levels of sleep staging accuracy,60 and correlation to PSG evaluations.61 Ballistocardiography-based sensing pads can automatically stage sleep comparatively to PSG or ECG.48,62 Finally, there are consumer Electroencephalography (EEG) headsets that have similar accuracy to PSG.63 Implementation of these passive measures to track abundant amounts of sleep data would allow one to capture the subtle, long term sleep deviations that could be indicative of AD related changes long before more blatant cognitive symptoms manifest.
Neuropsychiatric behavioral disruptions. Sensors: GPS, IMU, Device Usage Log
Beyond declines in specific cognitive and physiological domains, Alzheimer’s disease has also been associated with wider-range disruptions of behavior. Approximately 90% of Alzheimer’s patients experience at least one neuropsychiatric symptom64 with a spectrum of resulting behavioral changes such as mood disruptions, agitation and apathy.65 Apathy is one of the most common disruptions, affecting up to 90% of patients66 and has been implicated in patients’ lessened ability to carry out activities of daily living as well as a decreased motivation to participate in social activities.64,66 Depressive features are also common with up to 25% of patients being diagnosed with major depression and 50% experiencing depressive symptoms.67,68 Social withdrawal and dysphoria can precede diagnosis by years69 and are commonly seen in MCI populations as an early manifestation of AD.70 These depressive symptoms are also implicated in the decreased ability to perform activities of daily living and disruption of patients’ routines.71 These changes in patient’s life activities manifest themselves in tangible ways, studies have shown decreases in the size of patients’ social networks and frequency of social contact.72 These neuropsychiatric disruptions cause early impairment to more complex activities of daily living and can precede the dementia phase.64 Similar decreases in time spent outside of the house73 and social network sizes72,74 were seen in MCI populations. While there was some dispute as to the nature of the relationship between social activity and cognition; some studies showing social activity at baseline was a predictor or risk factor of progression to dementia,72 and that the current level of social activity was correlated to current level of cognitive decline.75 Continuously monitoring these complex everyday activities may demonstrate the behavioral disruptions resulting from the earliest underlying neurobiological changes.
Depression and anxiety symptomatology
Recently, there has been increased interest in using passive data from smartphones and wearables in psychiatry and metal health applications with depressive symptoms being of key interest.76,77 Mobility features such as location variance in terms of time and location extracted from the GPS sensors of smartphones have been shown to correlate with depressive symptom severity as determined by questionnaire.78 Actigraphy data from the accelerometers on wrist-worn wearables were also able to passively distinguish between subjects with Major Depressive Disorder while passive actigraphy has also been found to be helpful in establishing an objective measure of apathy in diminished activity levels.79 Beyond passive measures of physical activity and location, the smartphone also allows for insight into social activity. Meta-information on text messages and conversation frequency have also been correlated with depression severity as well as providing an objective correlate to the Social Rhythm Metric in Bipolar Disorder.80 Specific types of smartphone use, non-social (e.g., news consumption) and social (e.g., social networking) have been correlated to anxiety and depressive symptom severity.81 While more research needs to be performed in the Alzheimer’s space, these studies indicate the viability of using passive smartphone use data to provide objective measures of subtle neuropsychiatric behavioral disruptions.
AD patients experience spatial confusion and get lost even when around familiar places, resulting in wandering behaviors. Although these symptoms present themselves at later stages of the disease, subtle changes in commuting or executive function patterns may be detected earlier. For example, driving pattern features such as reduced speed (at least 10mph slower) compared to the rest of the traffic, reduced—less than half—mileage overall and a relative increase in the proportion of mileage driven close to home was shown in subjects with early stage dementia.82,83 Other metrics include the repeatability of a particular commuting pattern, route tortuosity and the consistency of routine locations on a particular day of the week. A mathematical descriptor of habitual location patterns was proposed by Eagle and associates, by performing principal component analysis on the geo-location vectors and extracting Eigen Behaviors, a digital biomarker of location frequency and intensity84: longitudinal excursions from these patterns may indicate an onset of a preclinical manifestation of AD.
Executive function. Sensors: phone usage log, touchscreen
Alzheimer’s Disease has long been defined by its characteristic decline in cognitive function and thus been evaluated by neuropsychological measures of the following broad cognitive domains: Memory, Attention, Executive Function, Language and Visuospatial Memory.85 Although these impairments have traditionally been considered a late-stage phenomenon, there is increasing evidence that cognitive changes in these domains may occur decades before dementia,86 specifically memory and executive function,87 and attention.88 Traditionally, these neuropsychological evaluations have been performed as test batteries or active tasks, typically involving an administrator.89 The logistical burden of administering these tests as well as their susceptibility to practice effects21 preclude their widespread use in determining a preclinical individual baseline as well as their use in continuous sampling to detect longitudinal deviations. In response to these shortcomings, there has been recent interest in the concept of digital phenotyping and its applications to mental health and psychiatry.90
The application of digital phenotyping to neurodegenerative conditions has already shown promising results. Researchers have been able to detect users with neurodegenerative conditions such as Parkinson’s and Alzheimer’s from web search data.91 Research at digital health company Mindstrong (Palo Alto, CA) has shown that continuous data from seven days of passive smartphone interactions can predict performance on traditional assessments of memory, language, dexterity and executive function.92 However, digital phenotyping in neurodegenerative conditions is still in its infancy as groups have yet to establish a clear, functional link between these passive activities and the cognitive domains of interest. Nevertheless, there are certain passive digital scenarios and evolving associated metrics that appear to tie back to cognitive areas of interest. For example, task-switching or the ability to shift between multiple goals is a component of executive function.93 Human Computer Interaction studies have started examining passive user-specific app re-visitation rates94 as well as the time-overhead cost from switching between applications95 that can be considered a naturalistic example of task-switching. Vigilance, or the ability to sustain attention on a task, is a measure of overall attention70 and studies have been able to correlate level of alertness to temporal rhythms of application usage.96 While more work is required to further develop and validate these measures across different domains of interest, as well as to apply them in a longitudinal AD study setting, these examples show the potential of passively interrogating cognitive domains from continuous user data.
Future work: Alzheimer’s disease forecasting using multiple digital senses
Each device sensor data stream (e.g., IMU) can be used to define an overall neurological health metric (e.g., gait symmetry) for a particular domain (e.g., gross motor control or balance). To date, there exists a significant amount of individual sensor data→metric→domain→disease validation coming mainly from well-controlled, lab-based clinical observations, some of which are listed in this review article.
Similar metrics, acquired longitudinally and passively, in-the-wild (meaning not in a controlled lab setting), using consumer-grade wearable devices, could produce data that could lead to domain predictors of AD before the actual clinical manifestation of the disease. The overall predictor signal is weak, since the changes in each domain are slow and difficult to separate from normal decline of ageing. Yet, the promise is to further amplify the signal’s ability to forecast AD by combining multiple metrics in a multivariate scoring and, if possible, a detection system. A multivariate approach was recently used to quantify symptom severity in Parkinson’s Disease patients based on mobile devices signals, resulting in a disease severity scoring system.97 Given the long time required for AD symptoms to fully manifest, a scoring or classification system could operate by means of anomaly detection, i.e., between user longitudinal trajectories, or/and by means of supervised training. Both approaches would require longitudinal observational studies involving healthy control, converter (to AD) and confirmed MCI cohorts that allow feature extraction of metrics, to inform—or train—the scoring or classification algorithms. In order to establish ground truth in such studies, validation using existing disease assessment methods such as cognitive tests, genomic phenotyping, and ideally longitudinal imaging (volumetric MRI, amyloid PET imaging, tau PET imaging) would be required.
Given the staggering current and escalating projected costs for providing care to AD patients, consumer grade technologies able to detect and monitor, once diagnosed, AD progression represent an urgent need. When developed to full potential, one can envision digital phenotyping of AD becoming a digitally embedded routine practice, triggering a series of interventional measures.
One of the main questions that emerges with such forecasting systems is what to do when a signal is detected. While the debate on the preferred course of action is still on and it involves among others, regulatory, ethical, legal, data privacy and clinical considerations, some options involve:
-Notifying the user that there is something out of the normal with his/her longitudinal rate of progression of neurological health, so he/she can seek further clinical assessment.
-Providing longitudinal disease-related digital biomarkers to a healthcare practitioner, to allow for objective and continuous clinical evaluation of a user.
In parallel, such technologies can be used to establish objective, personalized baseline reference standards to design innovative clinical trials that assess the effectiveness of onset delaying or disease modifying treatment, once available.
An overwhelming amount of work lies ahead before we can claim forecasting and detection of Alzheimer’s disease especially in the preclinical phase, using consumer grade devices, passive data monitoring and analytics. It will require longitudinal, very large population observational studies, to account for inter and intra subject variability. It will also require new ways of securely managing and processing this vast amount of information. Underscoring the potential for such consumer digital devices to impact healthcare, the FDA recently issued guidelines98 to provide a clear path and encourage technology developers in their quest for efficient digital phenotyping.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Sperling, R., Mormino, E. & Johnson, K. The evolution of preclinical Alzheimer’s disease: implications for prevention trials. Neuron 84, 608–622 (2014).
Dorsey, E. R., Papapetropoulos, S., Xiong, M. & Kieburtz, K. The first frontier: digital biomarkers for neurodegenerative disorders. DIB 1, 6–13 (2017).
Doraiswamy, P. M., Narayan, V. A. & Manji, H. K. Mobile and pervasive computing technologies and the future of Alzheimeras clinical trials. npj. Digital Med 1, 1 (2018).
Tsanas, A., Little, M. A., McSharry, P. E., Spielman, J. & Ramig, L. O. Novel speech signal processing algorithms for high-accuracy classification of Parkinson’s disease. Ieee. Trans. Biomed. Eng. 59, 1264–1271 (2012).
Albers, M. W. et al. At the interface of sensory and motor dysfunctions and Alzheimer’s disease. Alzheimers Dement. 11, 70–98 (2015).
Verghese, J. et al. Gait dysfunction in mild cognitive impairment syndromes. J. Am. Geriatr. Soc. 56, 1244–1251 (2008).
Lyons, B. E. et al. Pervasive computing technologies to continuously assess Alzheimer’s disease progression and intervention efficacy. Front. Aging Neurosci. 7, 102 (2015).
Ellis, R. J. et al. A validated smartphone-based assessment of Gait and Gait variability in Parkinson’s disease. PLoS ONE 10, e0141694 (2015).
Buracchio, T., Dodge, H. H., Howieson, D., Wasserman, D. & Kaye, J. The trajectory of gait speed preceding mild cognitive impairment. Arch. Neurol. 67, 980–986 (2010).
Brach, J. S., Studenski, S., Perera, S., VanSwearingen, J. M. & Newman, A. B. Stance time and step width variability have unique contributing impairments in older persons. Gait. Posture 27, 431–439 (2008).
Case, M. A., Burwick, H. A., Volpp, K. G. & Patel, M. S. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA 313, 625–626 (2015).
Muro-de-la-Herran, A., Garcia-Zapirain, B. & Mendez-Zorrilla, A. Gait analysis methods: an overview of wearable and non-wearable systems, highlighting clinical applications. Sensors 14, 3362–3394 (2014).
Rabinowitz, I. & Lavner, Y. Association between finger tapping, attention, memory, and cognitive diagnosis in elderly patients. Percept. Mot. Skills 119, 259–278 (2014).
Chaibub Neto, E. et al. Personalized hypothesis tests for detecting medication response in parkinson disease patients using iPhone sensor data. Pac. Symp. Biocomput. 21, 273–284 (2016).
Austin, D. et al. Measuring motor speed through typing: a surrogate for the finger tapping test. Behav. Res. Methods 43, 903–909 (2011).
Stringer, G. et al. Can you detect early dementia from an email? A proof of principle study of daily computer use to detect cognitive and functional decline. Int. J. Geriatr. Psychiatry. 33, 867–874 (2018).
Stirling, L. A. et al. Use of a tracing task to assess visuomotor performance: effects ofage, sex, and handedness. J. Gerontol. A Biol. Sci. Med. Sci. 68, 938–945 (2013).
König, A. et al. Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 1, 112–124 (2015).
Fraser, K. C., Meltzer, J. A. & Rudzicz, F. Linguistic features identify Alzheimer’s disease in narrative speech. JAD 49, 407–422 (2015).
Pistono, A. et al. Pauses during autobiographical discourse reflect episodic memory processes in early Alzheimer’s disease. JAD 50, 687–698 (2016).
Goldberg, T. E., Harvey, P. D., Wesnes, K. A., Snyder, P. J. & Schneider, L. S. Practice effects due to serial cognitive assessment: implications for preclinical Alzheimer’s disease randomized controlled trials. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 1, 103–111 (2015).
Crutcher, M. D. et al. Eye tracking during a visual paired comparison task as a predictor of early dementia. Am. J. Alzheimers Dis. Other Demen. 24, 258–266 (2009).
Fotiou, D. F. et al. Cholinergic deficiency in Alzheimer’s and Parkinson’s disease: evaluation with pupillometry. Int. J. Psychophysiol. 73, 143–149 (2009).
Fernández, G. et al. Eye movement alterations during reading in patients with early Alzheimer's disease. Invest. Ophthalmol. Vis. Sci. 54, 8345–8348 (2013).
Yang, Q. et al. Long latency and high variability in accuracy-speed of prosaccades in Alzheimer’s disease at mild to moderate stage. Dement Geriatr. Cogn. Disord. Extra 1, 318–329 (2011).
Ladas, A., Frantzidis, C., Bamidis, P. & Vivas, A. B. Eye Blink Rate as a biological marker of mild cognitive impairment. Int. J. Psychophysiol. 93, 12–16 (2014).
Zola, S. M., Manzanares, C. M., Clopton, P., Lah, J. J. & Levey, A. I. A behavioral task predicts conversion to mild cognitive impairment and Alzheimer’s disease. Am. J. Alzheimers Dis. Other Demen. 28, 179–184 (2013).
Bierer, L. M. et al. Neurochemical correlates of dementia severity in Alzheimer’s disease: relative importance of the cholinergic deficits. J. Neurochem. 64, 749–760 (1995).
Prettyman, R., Bitsios, P. & Szabadi, E. Altered pupillary size and darkness and light reflexes in Alzheimer’s disease. J. Neurol. Neurosurg. Psychiatry 62, 665–668 (1997).
Fernández, G. et al. Patients with mild Alzheimer’s disease fail when using their working memory: evidence from the eye tracking technique. J. Alzheimers Dis. 50, 827–838 (2016).
Molitor, R. J., Ko, P. C. & Ally, B. A. Eye movements in Alzheimer’s disease. JAD 44, 1–12 (2015).
Pavisic, I. M. et al. Eyetracking metrics in young onset Alzheimer’s disease: a window into cognitive visual functions. Front. Neurol. 8, 332–316 (2017).
Thayer, J. F., Hansen, A. L., Saus-Rose, E. & Johnsen, B. H. Heart rate variability, prefrontal neural function, and cognitive performance: the neurovisceral integration perspective on self-regulation, adaptation, and health. Ann. Behav. Med. 37, 141–153 (2009).
Parvizi, J., Hoesen, G. W. V. & Damasio, A. The selective vulnerability of brainstem nuclei to Alzheimer's disease. Ann. Neurol. 49, 53–66 (2001).
Rüb, U. et al. The autonomic higher order processing nuclei of the lower brain stem are among the early targets of the Alzheimer’s disease-related cytoskeletal pathology. Acta Neuropathol. 101, 555–564 (2001).
Frewen, J. et al. Cognitive function is associated with impaired heart rate variability in ageing adults: the Irish longitudinal study on ageing wave one results. Clin. Auton. Res. 23, 313–323 (2013).
Hjortskov, N. et al. The effect of mental stress on heart rate variability and blood pressure during computer work. Eur. J. Appl. Physiol. 92, 84–89 (2004).
Quintana, D. S., Guastella, A. J., Outhred, T., Hickie, I. B. & Kemp, A. H. Heart rate variability is associated with emotion recognition: direct evidence for a relationship between the autonomic nervous system and social cognition. Int. J. Psychophysiol. 86, 168–172 (2012).
de Vilhena Toledo, M. A. & Junqueira, L. F. Cardiac autonomic modulation and cognitive status in Alzheimer’s disease. Clin. Auton. Res. 20, 11–17 (2010).
Piwek, L., Ellis, D. A., Andrews, S. & Joinson, A. The rise of consumer health wearables: promises and barriers. PLoS Med. 13, e1001953 (2016).
Jo, E., Lewis, K., Directo, D., Kim, M. J. & Dolezal, B. A. Validation of biofeedback wearables for photoplethysmographic heart rate tracking. J. Sports Sci. Med. 15, 540–547 (2016).
Parak, J. et al. Evaluation of the beat-to-beat detection accuracy of PulseOn wearable optical heart rate monitor. Conf Proc IEEE Eng Med Biol Soc. 2015, 8099–8102 (2015).
Schäfer, A. & Vagedes, J. How accurate is pulse rate variability as an estimate of heart rate variability? A review on studies comparing photoplethysmographic technology with an electrocardiogram. Int. J. Cardiol. 166, 15–29 (2013).
Tison, G. H. et al. Passive detection of atrial fibrillation using a commercially available smartwatch. JAMA Cardiol. (2018). https://doi.org/10.1001/jamacardio.2018.0136
Ballinger, B. et al. DeepHeart: semi-supervised sequence learning for cardiovascular risk prediction. arXiv:1802.02511 (2018)
Kinnunen, H. Sleep Lab validation of a wellness ring in detecting sleep patterns based on photoplethysmogram, actigraphy and body temperature (2016). at http://myzeo.xaoc-lab.ru/wp-content/uploads/2018/04/Validity-of-the-OURA-Ring-in-determining-Sleep-Quantity-and-Quality-2016.pdf.
Friedrich, D., Aubert, X. L., Führ, H. & Brauers, A. Heart rate estimation on a beat-to-beat basis via ballistocardiography—a hybrid approach. Conf Proc IEEE Eng Med Biol Soc. 2010, 4048–4051 (2010).
Migliorini, M. et al. Automatic arrhythmia detection based on heart beat interval series recorded through bed sensors during sleep. IEEE Computing in Cardiology 2011 38, 337–340 (2011).
Picard, R. W., Fedor, S. & Ayzenberg, Y. Multiple arousal theory and daily-life electrodermal activity asymmetry. Emot. Rev. 8, 62–75 (2015).
Newham, W. G. & Tayebjee, M. H. Excellent symptom rhythm correlation in patients with palpitations using a novel Smartphone based event recorder. J. Atr. Fibrillation 10, 1514 (2017).
Magnani, J. W. et al. The atrial fibrillation health literacy information technology system: pilot assessment. JMIR Cardio. 1, e7 (2017).
Vitiello, M. V., Prinz, P. N., Williams, D. E., Frommlet, M. S. & Ries, R. K. Sleep disturbances in patients with mild-stage Alzheimer’s disease. J. Gerontol. 45, M131–M138 (1990).
Bliwise, D. L. Sleep disorders in Alzheimer’s disease and other dementias. Clin. Cornerstone 6, S16–S28 (2004).
Moe, K. E., Vitiello, M. V., Larsen, L. H. & Prinz, P. N. Sleep/wake patterns in Alzheimer's disease: relationships with cognition and function. J. Sleep. Res. 4, 15–20 (1995).
Montplaisir, J., Petit, D., Lorrain, D. & Gauthier, S. Sleep in Alzheimer’s disease: Further considerations on the role of brainstem and forebrain cholinergic populations in sleep-wake mechanisms. Sleep 18, 145–148 (1995).
Simic, G. et al. Does Alzheimer’s disease begin in the brainstem? Neuropathol. Appl. Neurobiol. 35, 532–554 (2009).
Braak, H. & Del Tredici, K. Evolutional aspects of Alzheimer’s disease pathogenesis. JAD 33, S155–S161 (2013).
Lim, A. S. P., Kowgier, M., Yu, L., Buchman, A. S. & Bennett, D. A. Sleep fragmentation and the risk of incident Alzheimer’s disease and cognitive decline in older persons. Sleep 36, 1027–1032 (2013).
Lee, J. & Finkelstein, J. Consumer sleep tracking devices: a critical review. Stud. Health Technol. Inform. 210, 458–460 (2015).
Zambotti, M., de, Rosas, L., Colrain, I. M. & Baker, F. C. The sleep of the ring: comparison of the ŌURA sleep tracker against polysomnography. Behav. Sleep. Med. 0, 1–15 (2017).
Renevey, P. et al. Optical wrist-worn device for sleep monitoring. Joint Conference of the EMBEC & NBC 2017. 65, 615–618 (2017).
Kortelainen, J. M., Mendez, M. O., Bianchi, A. M., Matteucci, M. & Cerutti, S. Sleep staging based on signals acquired through bed sensor. IEEE Trans. Inf. Technol. Biomed. 14, 776–785 (2010).
Debellemaniere, E. et al. Performance of an ambulatory dry-EEG device for auditory closed-loop stimulation of sleep slow oscillations in the home environment. Front. Hum. Neurosci. 12, 88 (2018).
Gauthier, S. et al. Management of behavioral problems in Alzheimer's disease. Int. Psychogeriatr. 22, 346–372 (2010).
Mega, M. S., Cummings, J. L., Fiorello, T. & Gornbein, J. The spectrum of behavioral changes in Alzheimeras disease. Neurology 46, 130–135 (1996).
Landes, A. M., Sperry, S. D., Strauss, M. E. & Geldmacher, D. S. Apathy in Alzheimer's disease. J. Am. Geriatr. Soc. 49, 1700–1707 (2001).
Wragg, R. E. & Jeste, D. V. Overview of depression and psychosis in Alzheimer's disease. Am. J. Psychiatry 146, 577–587 (1989).
Payne, J. L. et al. Relationship of cognitive and functional impairment to depressive features in Alzheimer's disease and other dementias. J. Neuropsychiatry Clin. Neurosci. 10, 440–447 (1998).
Jost, B. C. & Grossberg, G. T. The evolution of psychiatric symptoms in Alzheimer’s Disease: a natural history study. J. Am. Geriatr. Soc. 44, 1078–1081 (1996).
Oken, B. S., Salinsky, M. C. & Elsas, S. M. Vigilance, alertness, or sustained attention: physiological basis and measurement. Clin. Neurophysiol. 117, 1885–1901 (2006).
Lyketsos, C. G. et al. Major and minor depression in Alzheimer’s disease: prevalence and impact. J Neuropsychiatry Clin Neurosci. 9, 556–561 (1997).
Holtzman, R. E. et al. Social network characteristics and cognition in middle-aged and older adults. J Gerontol B Psychol Sci Soc Sci. 59, 278–284 (2004).
Kaye, J., Matteck, N., Hayes, T., Austin, D. & Dodge, H. Community life withdrawal with MCI progression. Alzheimer’s & Dementia 8, 483 (2012).
Bennett, D. A., Schneider, J. A., Tang, Y., Arnold, S. E. & Wilson, R. S. The effect of social networks on the relation between Alzheimer’s disease pathology and level of cognitive function in old people: a longitudinal cohort study. Lancet Neurol. 5, 406–412 (2006).
Krueger, K. R. et al. Social engagement and cognitive function in old age. Exp. Aging Res. 35, 45–60 (2009).
Torous, J. & Powell, A. C. Current research and trends in the use of smartphone applications for mood disorders. Internet Interventions 2, 169–173 (2015).
Insel, T. R. Digital phenotyping: technology for a new science of behavior. JAMA. 318, 1215–1216 (2017).
Saeb, S., Lattie, E. G., Schueller, S. M., Kording, K. P. & Mohr, D. C. The relationship between mobile phone location sensor data and depressive symptom severity. PeerJ. 4, e2537 (2016).
Kluge, A. et al. Combining actigraphy, ecological momentary assessment and neuroimaging to study apathy in patients with schizophrenia. Schizophr. Res. 195, 176–182 (2018).
Abdullah, S. et al. Cognitive rhythms: unobtrusive and continuous sensing of alertness using a mobile phone. UbiComp ’16 Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 178–189 (2016).
Elhai, J. D., Dvorak, R. D., Levine, J. C. & Hall, B. J. Problematic smartphone use: a conceptual overview and systematic review of relations with anxiety and depression psychopathology. J. Affect Disord. 207, 251–259 (2017).
Eby, D. W., Silverstein, N. M., Molnar, L. J., LeBlanc, D. & Adler, G. Driving behaviors in early stage dementia: a study using in-vehicle technology. Accid. Anal. Prev. 49, 330–337 (2012).
Babulal, G. M. et al. Creating a driving profile for older adults using GPS devices and naturalistic driving methodology. F1000Res. 5, 2376–2318 (2016).
Eagle, N., Pentland, A. S. & Lazer, D. Inferring friendship network structure by using mobile phone data. Proc. Natl. Acad. Sci. U. S. A. 106, 15274–15278 (2009).
Swainson, R. et al. Early detection and differential diagnosis of Alzheimer’s disease and depression with neuropsychological tasks. Dement. Geriatr. Cogn. Disord. 12, 265–280 (2001).
Mortamais, M. et al. Detecting cognitive changes in preclinical Alzheimer’s disease: a review of its feasibility. Alzheimers Dement. 13, 468–492 (2016).
Perry, R. J. & Hodges, J. R. Attention and executive deficits in Alzheimer’s disease: a critical review. Brain. 122, 383–404 (1999).
Tales, A., Muir, J. L., Bayer, A., Jones, R. & Snowden, R. J. Phasic visual alertness in Alzheimer’s disease and ageing. Neuroreport. 13, 2557–2560 (2002).
Strauss, E., Sherman, E. M. S. & Spreen, O. A compendium of neuropsychological tests: administration, norms, and commentary. (Oxford University Press, 2006).
Glenn, T. & Monteith, S. New measures of mental state and behavior based on data collected from sensors, smartphones, and the Internet. Curr. Psychiatry Rep. 16, 523 (2014).
White, R. W., Doraiswamy, P. M. & Horvitz, E. Detecting neurodegenerative disorders from web search signals. npj Digital Med. 1, 8 (2018).
Dagum, P. Digital biomarkers of cognitive function. npj Digital Med. 1, 10 (2018).
Hofmann, W., Schmeichel, B. J. & Baddeley, A. D. Executive functions and self-regulation. Trends in Cog. Sci. 16, 174–180 (2012).
Jones, S. et al. Revisitation analysis of smartphone app use. UBICOMP ’15 (2015).
Leiva, L., Böhmer, M., Gehring, S. & Krüger, A. Back to the App: the costs of mobile application interruptions. Mobile HCI ’12 (2012).
Murnane, E. L. et al. Mobile manifestations of alertness: connecting biological rhythms with patterns of smartphone App use. Mobile HCI ’16 (2016).
Zhan, A. et al. Using smartphones and machine learning to quantify Parkinson disease severity. JAMA Neurol. 75, 876–5 (2018).
FDA Digital Health Innovation Action Plan. 1–8 (2017). at https://www.fda.gov/downloads/MedicalDevices/DigitalHealth/UCM568735.pdf.
Thayer, J. F., Åhs, F., Fredrikson, M., Sollers, J. J. & Wager, T. D. A meta-analysis of heart rate variability and neuroimaging studies: Implications for heart rate variability as a marker of stress and health. Neurosci. Biobehav. Rev. 36, 747–756 (2012).
Vitiello, M. V. & Borson, S. Sleep disturbances in patients with Alzheimer’s disease. Mol. Diag Ther. 15, 777–796 (2001).
G.B.J. is supported by a grant from the National Center for Advancing Translational Sciences [NIH UL1 TR002544].