Smartphones offer a flexible tool to collect data about mental health, but less is known about their effectiveness as a method to assess variability in children’s problem behaviors. Caregivers of children with autism completed daily questions about irritability, anxiety and mood delivered via smartphones across 8-weeks. Smartphone questions were consistent with subscales on standard caregiver questionnaires. Data collection from 7 to 10 days at the beginning and 7 to 10 days at the end of the study were sufficient to capture similar amounts of variance as daily data across 8-weeks. Other significant findings included effects of caregiver socioeconomic status and placebo-like effects from participation even though the study included no specific treatment. Nevertheless, single questions via smartphones collected over relatively brief periods reliably represent subdomains in standardized behavioral questionnaires, thereby decreasing burden on caregivers.
There is increasing demand for quick, real-time caregiver and self reports on smartphones to replace longer symptom assessment questionnaires as outcome measures of environments beyond clinics in psychiatry and pediatric medicine.1,2 In developmental populations, smartphones have the potential to capture variability in children’s mood or challenging behaviors but many unknowns exist about implementation outside the clinic.
The goal of the present study was to identify effective methods to scale smartphones in the home to collect data from caregivers about children’s behavior. Daily smartphone delivery of questions about problem behaviors to caregivers of children with autism was tested across 8-weeks. No specific intervention was offered; therefore, significant changes in children’s symptoms over time were not predicted; rather we expected fluctuations commonly associated with autism.
Smartphone questions addressing irritability, mood, disruptive behavior, and anxiety were compared to similar subscales on standard caregiver questionnaires (Table 1; Supplementary Table 1). Over 8-weeks, caregivers reported improvement in their child’s mood, irritability and disruptive behaviors during treatment as usual.3 More educated caregivers rated their children’s irritability, disruptive behavior and anxiety, but not mood, more severely than less educated caregivers.
Higher total variance (TV), an overall measure of variability, on smartphone questions that targeted mood, irritability and disruptive behavior corresponded to more severity on standard paper questionnaires. Increased variance on one smartphone question was associated with increased variance on another (Supplementary Figure 1). As illustrated in Fig. 1, 7–10 days at the beginning and 7–10 days at the end of the study (truncated total variance) were sufficient to capture similar amounts of variance as 8-weeks of daily data (total variance).
Caregivers assessed their child’s behavior at home via questions on a smartphone. Single smartphone questions captured day-to-day variation in mood, anxiety and irritable/disruptive behaviors in children with autism, as indicated by significant relationships between smartphone questions and standardized questionnaires.
Reports from 7 to 10 days at the beginning and end of the study were equally valuable as 8-weeks of daily data. More educated caregivers rated their child’s behaviors more severely, indicating the importance of controlling for socioeconomic background. In addition, as occurred with the questionnaires, caregivers reported symptoms improving over time without explicit treatment.3 Frequent reminders from study staff to increase caregiver participation were necessary. Replication findings in a more diverse sample will be important. Nevertheless, single questions on smartphones over 7 to 10 days at the beginning and end of a study offer the potential to represent irritability, mood, anxiety and disruptive behavior in children as effectively as questionnaires or daily data collection over 8-weeks.
In total 20, 5–13 year-old participants with autism and their families were recruited through the Center for Autism and the Developing Brain (CADB) in White Plains, NY. Caregivers gave written consent; when possible, children 7 years and above assented. Weill Cornell Medicine’s IRB approved the study (#1405015095).
Participants and their caregivers completed an 8-week study consisting of home use of smartphones and clinic visits in weeks 1,4 and 8.3 In total 14 mothers, 3 fathers, 2 mothers & fathers, and 1 other family member completed the study. 55% had graduate degrees, 30% bachelor’s degrees, 5% some college and 10% high school diplomas.
Caregivers completed the Aberrant Behavior Checklist (ABC),4 Child Behavior Checklist (CBCL),5 Positive Affective and Negative Affective Scale (PANAS),6 Visual Analogue Scale (VAS) for anxiety and disruptive behavior during week 1 (T1) and week 8 (T2).
A designated caregiver installed the Janssen Autism Knowledge Engine (JAKE™) application during the first clinic visit.7 During weeks 1, 4 and 8, caregivers completed smartphone questions every day; during remaining weeks they answered questions at least 3 times per week. Questions addressed the child being tense/worried, irritable, and disruptive on an 8-point scale (0-very to 7-not at all) at the moment.
The internalizing subscale of the CBCL, the hyperactivity and irritability subscales of the ABC, the positive and negative mood scales of the PANAS and VAS for anxiety and disruptive behaviors were collected at T1 as representative of well documented standardized questionnnaires measuring roughly equivalent concepts. There were 456 responses on average per smartphone question. Mixed-effect ordinal logistic regression models assessed the consistency between caregiver report on the smartphone and questionnaires, with a subject-level random effect. A separate analysis included Day (1-up to 65), with a second analysis including Caregiver Education in the regression models. The clmm function from the ordinal package in R was used to fit the models (see Table 1 and Supplementary Table 1 for a summary of the regression models).
Total variance (TV) was computed for ordinal values for each smartphone question for each participant. TV was then calculated with only the first and last m days of data for a range of values of m. The correlation between TV and TV with data truncated at m day was computed for the range of truncation values m (Fig. 1). Pearson correlations, r, were computed between TV on the smartphone and paper questionnaires (Table 1).
We will share the data upon reasonable request.
Arean, P. A., Ly, K. H. & Andersson, G. Mobile technology for mental health assessment. Dialog. Clin. Neurosci. 18, 163–169 (2016).
Mohr, D. C., Zhang, M., & Schueller, S. M. Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annu. Rev. Clin. Psychol. 13, 23–47 (2017).
Jones, R. M., Carberry, C., Hamo, A., & Lord, C. Placebo-like response in absence of treatment in children with Autism. Autism Res. 10, 1567–1572 (2017).
Aman, M. G. & Singh, N. N. Aberrant Behavior Checklist (ABC). (Slosson Educational Publications, 1986).
Achenbach, T. M. & Rescorla, L. Manual for the ASEBA School-age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families (2001).
Watson, D., Clark, L. A. & Tellegen, A. Development and Validation of Brief Measures of Positive and Negative Affect - the Panas Scales. J. Pers. Soc. Psychol. 54, 1063–1070, https://doi.org/10.1037/0022-35126.96.36.1993 (1988).
Ness, S. L. et al. Multimodal data capture system: insights from an observational study of autism spectrum disorder. Front. Neurosci. 11, 517 (2017).
Lord, C. et al. Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) Manual (Part 1): Modules 1–4. (Western Psychological Services, 2012).
Rutter, M., LeCouteur, A. & Lord, C. Autism Diagnostic Interview-Revised (ADI-R) Manual. (Western Psychological Services, 2003).
Robinson, B. C. Validation of a caregiver strain index. J. Gerontol. 38, 344–348 (1983).
We acknowledge our funding sources SFARI 336363 and SFARI 391635.
Catherine Lord receives royalties from the ADOS and ADI-R, and all proceeds are donated to charity. All other authors have no financial or non-financial competing interests.
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
About this article
Cite this article
Jones, R.M., Tarpey, T., Hamo, A. et al. Smartphone measures of day-to-day behavior changes in children with autism. npj Digital Med 1, 34 (2018). https://doi.org/10.1038/s41746-018-0043-3
Data-Driven Diagnostics and the Potential of Mobile Artificial Intelligence for Digital Therapeutic Phenotyping in Computational Psychiatry
Biological Psychiatry: Cognitive Neuroscience and Neuroimaging (2019)