Introduction

Population health approaches to the management of major societal healthcare challenges have received much attention of late, with the rise of big data and advances in health assessment technology. Population level preventative screening has the potential to significantly reduce costs and improve long-term outcomes for a variety of conditions. Globally, falls are the most frequent cause of accidental death and disability, thought to cost $50bn each year in the United States1 and €25bn each year in the European Union.2 Falls can be underreported, with the first presentation of a patient to clinical services only after the first fall has occurred.3 Extensive research has shown that falls are not inevitable and can be reduced by 20–50% by appropriate early intervention.4,5 However, effective early intervention is contingent on accurate screening and identification of those people at risk of experiencing a fall, ensuring timely and appropriate referral of those patients at higher risk.

Assessment of functional mobility is a critical element in quantifying and understanding the nature of falls risk, and in measuring response to remedial interventions in the assessment and management of falls risk and frailty.6 A variety of functional mobility assessments are available and commonly used in research; they include the Timed Up and Go (TUG) test,7 which has been shown to be a valid and reliable functional test.8 Persons taking longer to complete the TUG test have been shown to have high rates of falls, frailty, and other clinical conditions.9 However, research has shown the TUG test has only limited accuracy in predicting falls10,11 in older adults. Similarly, slow walking speed has been found to have strong negative associations with a number of long-term outcomes such as dementia, frailty, and longevity.12

Objective assessment of falls risk, frailty, and mobility, using a reliable functional test combined with wearable sensors and predictive algorithms supports the assessment by lower cost and lower skilled personnel or clinical personnel without specialist expertise, and may help improve the precision of clinical assessment and referral; this can increase the efficiency of primary care by using standardized assessments that can be administered by a healthcare assistant, as well as reducing inappropriate referrals. Moreover, deployment of community screening technologies may reduce unnecessary outpatient visits and health care utilization by driving medium and low risk patients away from hospitals towards community falls prevention activities. Such an approach could reduce the instances of escalation and emergency acute admissions through increased prevention arising from improved screening, triage, and onward referral of at-risk patients.

A number of recent studies have examining depth camera13 and sensor-based14,15,16,17,18,19,20 systems for assessing gait, mobility and fall risk in older adults. Sensor-based methods for assessing fall risk are often use prescribed motor tasks (e.g. TUG or Five Time Sit to Stand), instrumented with inertial sensors. In a majority of studies, each patient’s self-reported history of falling (cross-sectional fall data) prior to assessment16,19 is used to validate the reported algorithm, while for a subset of studies each patient was followed-up and their falls tracked for a period of time after the assessment is performed; these fall data were then used to validate the methods’ performance in predicting falls (prospective fall data)14,20,21. Previous research from our group22 has reported that fall history data were more accurate in classifying healthy controls than 2 years prospective data (incidents of falls during a follow-up period). Recent studies23,24 have pointed out flaws in the methodologies of many reported studies which aim to assess fall risk and/or predict falls. Specifically, they noted insufficient validation and inappropriate use of statistical modeling methods in many of the studies reported to date.

The goal of this research is to report results for the deployment in routine clinical practice of a novel digital assessment protocol and algorithm for prediction of falls, frailty and mobility impairment; developed, trained and validated by our team over the past 11 years. We report an observational study of the performance of these algorithms, tested on a large statistically independent sample, in diverse real-world clinical settings over a period of 5 years.

Results

A total of 14,611 records from 8521 participants (5415 female, 3106 males) were available for analysis. Where multiple records were available per participant, only the first record was included in analysis. The mean age of the sample was 72.7 ± 10.7 years, with mean height and weight of 165.9 ± 10.4 cm and 73.8 ± 15.9 kg, respectively. The mean TUG time was 13.9 ± 7.4 s, while the mean gait velocity (during the TUG test) was 101.9 ± 32.5 cm/s (see Table 1). Figure 1 shows the distributions of the age, TUG time and gait velocity for the cohort.

Fig. 1
figure 1

Histograms of age (first panel), TUG time (second panel) and gait velocity (third panel) for all participants included in the sample.

Data were obtained from 38 healthcare organizations from six countries (UK, USA, Ireland, Canada, Australia and Hungary). The breakdown of the sample of participants per country is as follows: UK—29.5%, Ireland—41.8%, USA—11.1%, Canada—15.2%, Others—2.2%. Organizations were divided into five categories as follows: Academic (3.2%), Primary care (91.4%), Residential care (1.3%), Secondary care (<1%), Senior living (3.2%). Users conducting each patient assessment for each organization were grouped into four categories as follows: care assistant (40.3%), Physician (<1%), physical therapist (55.9%) and researcher (3.0%). Similarly, the patient profiles assessed by each organization were grouped into six categories as follows: community dwelling older adults (88.3%), neurological patients (1.4%), rehabilitation patients (7.1%), residential care (2.0%), and unknown patient type (1.2%).

Questionnaire data

A total of 11,635 questionnaire records from 6950 participants were available for analysis, as above only the first record with a completed questionnaire for each participant was included in analysis. 1779 participants (25.6%) reported a history of falls in the previous 12 months with 4477 falls reported in total (average of 0.65 falls per participant and 2.51 falls per faller). 855 participants reported recurrent falls. Table 2 details the prevalence of each of the reported clinical risk factors per patient group and organization type.

Table 1 Demographic information for the sample (N = 8,521), broken down by patient type.

Polypharmacy was present for 3240 participants (46.6%), while mobility problems were reported by 2346 participants (33.7%). Vision problems were reported in 2262 records (32.5%), while dizziness while standing up (orthostatic hypotension) was reported by 547 participants (7.9%). A change in ability to manage routine activities in the home (a marker of a person’s ability to live independently) was reported by 1352 participants (19.4%).

Falls Risk Estimate

For the sensor based FRE (FREsensor), 554 of 8521 participants were excluded due to missing data, leaving 7,967 valid participant records for analysis. Missing data arose as the FRE is not calculated for participants under 60 years of age or if “Falls mode” on the software is turned off. Similarly, for FREcombined and FREclinical, 6,543 participants were included in the analysis, missing data arose for participants under 60 (no value calculated) and when the user chose to skip the questionnaire. The lower limit for age in the calculation of the FRE and FE is enforced due to the fact that the classifier models included in the software were trained and validated on a cohort of persons 60 years of age and older. Table 4 details the prevalence of high falls risk per organization type and patient type.

6295 participants had both valid FRE data and completed questionnaire data for the first record. 4693 of these participants reported no falls in the previous 12 months, 1008 (21.5%) of these were found to be at high risk or very high risk of falls (based on FREcombined). Conversely, 745 of these participants reported two or more falls in the previous 12 months, 153 (20.5%) of these participants were found to be at low risk of falls. FREcombined was found to be significantly associated with falls history (F = 214.19, p < 0.0001), see Fig.2.

Fig. 2: Association of falls risk score (FREcombined) with falls history (F = 214.19, p < 0.0001).
figure 2

The falls risk scores were grouped by no falls in the past 12 months, one fall in the past 12 months and two or more falls in the past 12 months. For each box, the central mark indicates the median, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, while outliers are denoted individually by ‘ + ’.

Frailty estimate

Valid records for 7536 participants were available for FEsensor, missing data arose as this measure was introduced in a later software version (2015) and so were not calculated for all participants over the study period. Similarly, no FE values were calculated for participants under 60 or if “Falls mode” was turned off. FEclinical and FEcombined were introduced in 2017, this fact along with the ability of the user to skip the questionnaire meant there were valid records for only 3016 participants for these measures. Table 3 details the prevalence of participants in each risk category for each of the FRE and FE scores by participant and organization type.

Table 2 Prevalence (%) of clinical fall risk factor per organization type and patient type (N = 6950).

2946 participants had both valid FE data and completed questionnaire data for their first record. 2143 of these participants reported no falls in the previous 12 months. 709 (33.1%) of those who reported no falls were found to be at high risk or very high risk of falls (based on FREcombined). Conversely, 387 participants reported two or more falls in the previous 12 months, 79 (20.4%) of these participants were found to be at low risk of falls. FEcombined was found to be significantly associated with falls history (F = 51.08, p < 0.0001).

Mobility impairment scores

The mobility impairment scores were introduced in 2017 and so values were not calculated for all participants over the study period. From the original 14,611 records for 8521 participants, there were valid mobility impairment scores for 5195 participants. Table 4 details the prevalence of mobility impairment per organization and patient type for each mobility impairment score. Each mobility score was significantly associated with falls history (speed: F = 76.78, p < 0.0001, turn: F = 41.47, p < 0.0001, transfers: F = 115.46, p < 0.0001, symmetry: F = 4.13, p < 0.05, variability: F = 6.23, p < 0.01. Four of five mobility scores were significantly associated with self-reported mobility problems (speed: F = 261.49, p < 0.0001, turn: F = 289.04, p < 0.0001, transfers: F = 547.07, p < 0.0001, symmetry: F = 3.24, p = 0.07, variability: F = 9.17, p < 0.01).

Table 3 Proportion of participants in each risk category for each of the Falls Risk Estimate (FRE) and Frailty Estimate (FE) scores.

Discussion

To the best of our knowledge this is the largest study reported to date (N = 8,521) on either objective measures of falls risk and frailty, or wearable sensor measures of gait and mobility. We report an observational study of a novel digital fall risk assessment protocol that supports the early identification of older adults at risk of falls and provides indications to inform the creation of appropriate personalized interventions to prevent falls. The classifier models used in this protocol were trained prior to deployment on an independent data set. The methods used have been previously validated and shown to be valid,25,26 reliable27,28 and more accurate than standard methods, in assessing falls21,29,30,31 and frailty.32,33

A large proportion of patients (range of 22.5–45.8% across measures) were predicted to be at high risk or very high risk for a measure of falls and frailty. These results agree with the epidemiological finding of 1 in 3 older adults being at risk of falling per year.34 As expected, a higher prevalence (50.4%) of high or very high falls risk was predicted in residential care settings when compared to primary care and senior living. Similarly, a higher prevalence (46.4%) of high or very high falls risk was predicted in residential care patients when compared to community dwelling older adults.35,36 25.6% of participants reported a fall in the previous 12 months, while 26.4% of participants were predicted to be at high or very high risk of falls (based on FREcombined), with 37.1% of participants being at high or very risk of frailty (FEcombined); this is lower than some international estimates suggest and may reflect the fact that the sample was: (a) healthier than the general population or; (b) fall data were unreliable due to difficulties with underreporting, self-reported recall3 or cognitive impairments.37 21.5% and 33.1% of participants who did not report any falls in the previous 12 months were predicted to be at high risk of falls and frailty respectively, which could suggest these are participants with undetected risk who could stand to benefit most from preventative screening programs. 20.5% and 20.4% of participants who reported two or more falls in the previous 12 months were found to be at low risk of falls and frailty, respectively. These participants might be more complex cases, representing a limitation on the accuracy of the assessment tool, where the source of falls risk is not adequately identified by the digital fall risk assessment protocol.

Polypharmacy is considered one of the strongest clinical risk factors for falls38 and was reported by 46.6% of participants overall, which is almost twice the rate of self-reported falls, with higher rates among those in residential care (52.7%) and senior living (55.1%). Mobility problems were reported by 33% of participants, while low speed (high risk speed score) was observed in 30% of participants. Higher rates (>50%) of those in senior living and secondary care reported mobility problems as an issue, with a larger proportion of high-risk speed scores observed in residential and secondary care.

We report an average TUG time of 13.9 ± 7.4 s, and an average gait velocity of 102.9 ± 32.5 cm/s. Both measures are in agreement with average values reported in the literature for this population.10,11,39 It is worth noting that the proportion of participants with impaired or high-risk symmetry is 2.7%, while 29.8% of patient records indicated slow speed (as calculated against a statistically independent reference data set). This might suggest that asymmetric gait is less prevalent than slow walking speed or high gait variability (19.5%), which is associated with falls.14 Problems with transfers (e.g. getting out of a chair), which is associated with poor lower limb and core strength40 were found in 17.5% of participants, problems turning (associated with poor balance or vestibular problems41) were found in 13.9%. The prevalence of slow walking speed (as determined by the relevant age and gender referenced mobility score) in residential care patients was 35.1% as compared to 18.1% in primary care and 4.8% in senior living. Understanding factors associated with fall risk, such as gait variability or slow walking speed may improve fall risk prevention interventions targeted to specific problems leading to future falls, mobility impairment or frailty.42

The reported data have limitations which we have attempted to address; data are from diverse real-world routine clinical settings and so were not controlled research-grade data, data were collected from a cross-section of the population, including population samples which might not be included in a typical research project. As such, some data were missing or were noisy, however, we do not believe there was a systematic or consistent reason for missing data that could affect the results. All users were provided with instruction on test protocol, but we have no way to evaluate treatment fidelity in the routine clinical practice for 38 organizations. We attempted to address this by performing outlier and artifact rejection on the data. In addition, all questionnaire data were self-reported (albeit mediated by the user supervising the delivery of the test and questionnaire). Due to the nature of the population, self-reported data on falls, particularly using retrospective falls history, can be under-reported. While 12 month falls history is a strong predictor of future falls,43 consensus guidelines44 suggest it is not an appropriate outcome measure for studies of falls and for this reason we have only reported associations with falls history rather than using it to report the predictive accuracy of the reported measures in this independent sample.

This study aims to show the behavior of these previously validated objective measures of falls risk, frailty and mobility impairment obtained from wearable sensor data, on a statistically independent data set from routine clinical settings. We also confirm their expected association with self-reported falls history and self-reported mobility problems.

These data are of crucial importance to clinicians interested in fall prevention as they provide a reliable basis for determining a patient’s falls risk and establishing preventative measures. This protocol which has been shown elsewhere to be cost-effective45 for use in fall prevention, broadens the base of users that can perform fall risk assessments. This is because it enables lower skilled or lower-cost staff, such as home care and community workers to perform reliable fall risk assessments in non-clinical environments and allows clinicians such as primary care physicians and physical therapists to interpret these data and improve the precision of referral and diagnosis. In addition, they may allow healthcare managers to estimate demand for services based on target population and expected prevalence of each risk category. This has the potential to improve patient outcomes and significantly reduce the financial burden imposed by falls on global healthcare systems.

Methods

A sample of 8521 participants were assessed using a digital fall risk assessment protocol (see Table 1). Data included measures of falls, frailty and mobility and were collected between 2014 and 2019. Data were anonymized and stored centrally on a cloud-based server. Each data record contained wearable sensor data and questionnaire responses on clinical fall risk factors.

Each participant provided informed consent, a supervising user consented the participant and collected the data using Kinesis QTUG™ (Kinesis Health Technologies, Dublin, Ireland), a Class I medical device, registered with US, Canadian, EU and Australian competent authorities. Each supervising user was provided with a user guide, an instructional video and additional training if required. Data were acquired in different contexts and pooled for the purposes of the analyses presented in this paper. Some data were obtained as part of research studies which had been approved by the relevant ethics committees and shared with the authors with the consent of study participants. Other data were collected as part of normal clinical practice from a variety of healthcare organizations (including primary care, senior living, secondary hospital care, and residential care) and shared with the authors in adherence with local data governance procedures. Data were anonymized and stored in line with data privacy regulations in each country (e.g. HIPAA, GDPR). Data were obtained from a variety of healthcare organizations including primary care, senior living, secondary (hospital) care, academic research and residential care. Users included individuals from a variety of professional groups, including physical therapists, care assistants and researchers. Each participant carried out the TUG test wearing body-worn inertial measurement units (IMUs) on each leg below the knee. Each IMU was held in place with dedicated reusable Velcro straps or with disposable bandages. Users were asked to instruct participants to complete the TUG test “as fast as safely possible”; participants were asked to stand-up, walk 3 m, turn 180°, walk back to the chair and sit down. Each test takes approximately 5 min, including application of the sensors and explaining the protocol. Data were extracted from the database using MySQL workbench 8.0 (Oracle, CA) and analyzed using Matlab v9.3.0 (Mathworks, Natick, VA).

Questionnaire data

The supervising user was asked to complete a questionnaire with the participant, asking them questions on their clinical fall risk factors (this questionnaire was optional and could be skipped in the software (for users for whom clinical falls data were not of interest)). The questionnaire (see Supplementary methods) is based on the American and British Geriatric societies (AGS/BGS) guidelines and aims to capture self-reported data on the main clinical risk factors linked to falls in older adults.46 The clinical risk factors recorded by the questionnaire include history of falls, polypharmacy, mobility problems, orthostatic hypotension, foot problems, vision problems (vision impairment) and ability to live independently. Participants were asked about their history of falls in the previous 12 months. Falls history was considered in three categories: no falls in the previous 12 months, one fall in the previous 12 month, two or more falls (recurrent faller) in the previous 12 months.

IMU sensor data

The recorded IMU sensor data are automatically processed immediately upon completion of each TUG test, producing 71 different measures of gait and mobility (QTUG mobility parameters), which are applied to pre-trained classifier models to produce validated statistical predictions of falls risk and frailty.29,30,31,32 For the purposes of this study we report a subset of these parameters: TUG time, gait velocity, Falls Risk Estimate (FRE), Frailty Estimate (FE), and mobility impairment scores. All data reported here were independent of the data previously used to develop and validate each of the statistical classifier models used (see Supplementary material for further details).

TUG time, the time to complete the TUG test, is a standard measure used to assess fall risk. It is measured from the moment the user says ‘go’ to the moment the participant’s back touches the chair. Traditionally, this is measured using a stopwatch but is captured here using a start/stop button on a tablet (QTUG software). The gait velocity during the TUG test is calculated using a previously reported algorithm.21,47 For reference purposes, we use a standard cut-off time for the TUG test; with a time greater than 13.5 s11 considered high risk.

Falls risk estimate

Statistical fall risk predictions are calculated using a combination of sensor data, anthropomorphic and questionnaire data applied to classifier models previously trained on independent data.29 If a full dataset is available for a participant, three falls risk estimate scores are produced: Sensor-based (FREsensor), Questionnaire-based (FREclinical) and Combined (FREcombined).

FREsensor is based on IMU sensor data and anthropomorphic data only, applied to a regularized discriminant model. FREclinical is based on questionnaire and anthropomorphic data applied to a logistic regression model. FREcombined is a weighted average of the other two FRE scores. (see Supplementary methods for more details). Prospective and cross-sectional validation data for each model have been reported previously29,30 and found that FREcombined is more accurate than both FREclinical and FREsensor individually, while FREcombined and FREsensor have also been shown to be significantly more accurate (approximately 20% depending on the study population) than the TUG time or Berg balance scale (BBS) in assessing falls risk.21,30 In this observational study, a one-way ANOVA was used to test the association between FREcombined and falls history.

Frailty estimate

As with the FRE, three frailty estimate (FE) scores—derived using previously validated predictive models for estimating frailty based on the Fried frailty phenotype48 (a well-validated standard measure of clinical frailty)—are calculated for each QTUG assessment: Sensor-based (FEsensor), Questionnaire-based (FEclinical), Combined (FEcombined).

FEsensor is based on a previously reported algorithm32 and uses IMU data and anthropomorphic data applied to a logistic regression model (see Supplementary material for more details). FEsensor was found to be slightly more accurate (4%) than TUG time in assessing frailty but would have the advantage of being faster to administer and suitable for a non-specialist user. FEclinical and FEcombined use a similar methodology to FREclinical and FREcombined29 which are based on the questionnaire data applied to a logistic regression model and a weighted average of the other two FE scores, respectively (see Supplementary methods for more details). Both are reported here for the first time. In this paper, a one-way ANOVA was used to test the association between FEcombined and falls history.

Mobility impairment scores

Mobility impairment is identified by grouping the calculated QTUG mobility parameters into five functional categories: Speed, Variability, Symmetry, Transfers, and Turning.

Mobility issues per functional category are identified by calculating a percentile score (compared to an independent reference sample22,30,31) for each QTUG parameter per group, with the mobility impairment score taken as the mean of the parameter percentiles within each functional category. Scores are also stratified by age range and gender. Each mobility impairment score can be interpreted as an aggregated measure (between 0 and 100%) of the severity of a person’s impairment in the given functional mobility category, referenced against the population stratified by age range and gender. See Supplementary methods for more details.

Table 4 Proportion of participants with high (or very high) falls risk (FREcombined, N = 6543), frailty (FEcombined, N = 3016) and mobility impairment (N = 5195) per QTUG score by organization and patient type.

Risk categories

The falls risk and frailty scores are essentially posterior probabilities and defined as low risk if less than 50%, 50–69% is considered medium risk, while high risk and very high risk are 70–89% and greater than 90% respectively. These probability definitions are based on empirically derived thresholds obtained from testing on algorithm development datasets.

A mobility impairment score of less than 50% is below the median (50th percentile) and considered low risk, while a score of 50–69% is deemed medium risk. If the mobility impairment score for a given category is above 70% (70th percentile) the participant is considered to display functional impairment in that functional category.

One-way ANOVA was also used to examine the association of mobility scores with falls history as well as with self-reported mobility problems.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.