KINECAL: A Dataset for Falls-Risk Assessment and Balance Impairment Analysis

The field of human action recognition has made great strides in recent years, much helped by the availability of a wide variety of datasets that use Kinect to record human movement. Conversely, progress towards the use of Kinect in clinical practice has been hampered by the lack of appropriate data. In particular, datasets that contain clinically significant movements and appropriate metadata. This paper proposes a dataset to address this issue, namely KINECAL. It contains the recordings of 90 individuals carrying out 11 movements, commonly used in the clinical assessment of balance. The dataset contains relevant metadata, including clinical labelling, falls history labelling and postural sway metrics. KINECAL should be of interest to researchers interested in the clinical use of motion capture and motion analysis.

, in his review of RGB + D datasets, detailed 45 datasets, 44 of which were recorded using Kinect.Of these datasets, only one, the K3Da dataset 7 had movements that are useful in the assessment of balance disorders, it has some metadata, but it is limited to age, height and weight.
Since the publication of the Firman paper 4 , a few datasets have been created to address these issues.The Multimodal Dataset 8 contains clinically relevant movements (Timed get-up-and-go (TUG), a single 30 second chair stand, a 45 second unilateral stance, and a 2 minute step test).However, the dataset comprises just 21 subjects, evenly split between young and old, and none of the participants had known movement impairment at the time of testing.The KIMORE Dataset 9 , has 78 subjects, 44 healthy (mean age 36.7 ± 16.8 years) and 32 suffering from motor dysfunction (mean age 60.44 ± 14.2 years).The dataset includes clinical scoring of the movements.However, the movements do not relate directly to falls-risk.Table 1 provides a comparison between KINECAL and other datasets, containing clinical movements.
KINECAL contains the recordings of 90 participants carrying out 11 movements, commonly used in the clinical assessment of balance impairment, frailty and falls-risk.Details of how each movement was carried out are shown in Table 2.The participants are grouped by both age and falls-risk.This enables researchers to disambiguate factors which relate to each.

Methods
Participants.90 participants, aged from 18 to 92, were recorded for the KINECAL dataset, carrying out a range of movements commonly used in clinical tests.The recordings were made using a Kinect V2.Further details of data collection are shown below.Ethical approval was obtained from the Manchester Metropolitan University Research Ethics Committee (ethics approval ref: 020517-ESS-CC (1), and ref: SE161757).All participants provided written informed consent.Participants were excluded if they had any of the following: • Treatment for cancer in the previous 2 years • Joint replacement in the previous year • Broken a leg, or hip bone or had a joint replaced, e.g.hip or knee, in the previous 2 years • Any lower limb amputation • Suffering from neuromuscular conditions (e.g.multiple sclerosis) • Have been diagnosed with Alzheimer's or Dementia • Cannot read and communicate in English, either verbal or written Clinically significant movements.Table 2 provides a list of the clinically significant movements included in the dataset, along with the instructions given to each participant.The movements are: Quiet standing, eyes open (EO) and eyes closed (EC), semi-tandem stance EO, tandem stance EO, 3 m walk and 5 STS-5, (this constitutes the Short Physical Performance Battery (SPPB) assessment).To assess a participant's ability to react to perturbation, the quiet standing trials EO/EC were repeated while standing on a compliant surface (Airex balance pad).Finally, two movements which are often used as standalone tests are included, i.e.TUG and unilateral stance EO/EC.

Data collection.
Recordings were made using a Kinect camera mounted on a tripod at a height of 1.14 m from the ground plane.Participants were asked to stand 3 m from the camera for static stances and 4 m for TUG, and 3 m walk (the additional meter was needed to capture the entire 3 m walk).Participants were recorded using custom software, skel recorder, written using Visual Studio 2015 and the Kinect SDK 2.0.skel recorder captures Depth and Skeleton data and stores it to hard disk.Figure 1 shows the equipment setup and Fig. 2 shows the output from the Kinect.This image was taken in laboratory conditions, but other recordings in KINECAL were made in more informal settings, such as church halls.
Quality control.Kinect can miss-classify structures, such as tables, which are in the frame along with the participants.If this happens, aberrant and truncated skeletons are included in the recordings.These types of artefacts reduce the quality of the dataset.In addition, some of the initial recordings were started after the movement had begun or stopped before the movement was completed.To ensure only good-quality recordings were included in the dataset, videos were created from the depth and skeletal data.These videos were reviewed manually.Any recordings with issues were excluded from the dataset.
Dataset structure.In designing KINECAL, previous studies relating to ageing and falls-risk were reviewed.
Some studies considered just age-related changes, e.g.young ( < 35 years old) vs older (≥65 years old) populations [10][11][12] .This polarisation is helpful if the question is, how does postural sway change with age.However, using this type of data alone can be problematic when applied to falls-risk.While ageing plays a part in falls-risk, it is not the whole story.For example, if we consider master athletes (individuals over the age of 30, but many of whom are aged over 80, who still take part in athletic competition).In spite of demanding regimes of training and competition, the occurrence of falls for master athletes was found to be no higher than it was for younger athletes 13 .While they are seen as functionally fitter when compared to those of a similar age who do not compete in sporting events, the effects of ageing are still detectable as changes in motor neurons 14,15 , but this does not translate to an increase in the number of falls.McPhee et al. 16 suggested that the protective effects of regular exercise are universal and not limited to elite athletes.An alternative approach, when considering falls-risk, is to control for age.This is the approach taken by [17][18][19] , who compared aged-matched populations of older fallers and non-fallers.However, this approach can make it just as challenging to understand the interaction of age and physical function.With these factors in mind, KINECAL is structured to help aid the separation of falls-risk from age effects.The following section details how participants were labelled to provide a range of groupings when working with the data.

STS-5
From a seated position, with your arms crossed over your chest, rise extending your legs fully, then sit down again.Repeat five times, as quickly as possible.

Quiet standing, Eyes open, Foam
Same instructions as for standing on a firm surface, but perform on a foam.

Quiet standing, Eyes closed, Foam
Same instructions as for standing on a firm surface, but perform on a foam.

Semi-tandem Stance
Stand with the toe of the back foot against the side of the heal of the front foot.Ether foot can be forward, which ever is most comfortable.The test is terminated after 20 seconds.

Tandem Stance
Stand with the toe of the back foot against the back of the heal of the front foot.Ether foot can be forward, which ever is most comfortable The test is terminated after 20 seconds.

Unilateral stance, Eyes open
Stand on one leg, whichever is most comfortable, with the other leg flexed 6 inches off the ground, hands by your side, eyes open.
The test is terminated after 20 seconds or when the lifted leg touches the ground.

Unilateral stance, Eyes closed
Stand on one leg, whichever is most comfortable, with the other leg flexed 6 inches off the ground, hands by your side, eyes closed.
The test is terminated after 20 seconds or when the lifted leg touches the ground.

TUG
From a seated position, stand, and walk to a marker 3 m away and return to the seat

m walk
From a standing position, walk to a marker 3 m away, and return to the seat Table 2.A description of movements in the recording of KINECAL, and how they were described to the participants.Based on the answer they gave and their age, participants were split into the following groups: • Healthy-Adult, members of this group, were all <65 years old and gave the answer None • Non-Faller, members of this group, were all ≥65 years old and gave the answer None • Self-reported-Faller, members of this group, were all ≥65 years old and gave the answer One, Two, Three, Four or more Table 3 details the numbers in each group, along with their age range and the gender split.
Single and multiple fallers.The Self-reported-Faller group can be further split into single and multiple fallers.
We make this distinction because someone who answered one to the falls history question might simply be the victim of bad luck and not someone with a high likelihood of future falls.Someone who declared they have fallen multiple times is more likely to suffer future falls and could be thought of as a true faller 21 .It is up to the individual researcher to make the best use of these labels.
• Self-reported-Faller_s, members of this group, were all ≥65 years old and gave the answer One • Self-reported-Faller_m members of this group, were all ≥65 years old and gave the answer Two, Three, Four or more Table 4 details the numbers in each group, along with their age range and the gender split.Note: the Self-reported-Faller_s and Self-reported-Faller_m groups are made up of members of the Self-Reported-Faller group.Participants are multipally labelled, which provides flexibility in use.However, it is up to the individual researcher to decide how to best use these labels.
Clinical labelling.The movements recorded in the dataset provide an alternative means of labelling participants.Moreover, one which relates to clinical tests of physical impairment, hence we call this clinical labelling.
Clinical labelling was a two-stage process.(1) The recordings for each participant were played back and marked against accepted thresholds for physical impairment and falls-risk, for each of the following tests: SPPB 22 ; Slow 3 m walk 23 ; TUG 24,25 and slow time to complete STS-5 26 .(2) Any individual categorised as impaired for a least two of these tests was labelled Clinically-At-Risk in the dataset.Using this method, six people were identified as belonging to the Clinically-At-Risk group.Details of this group are shown in Table 5.The thresholds of impairment for each test are discussed below.
Note: the Clinically-At-Risk group is made up of members of the Non-Faller group (2) and Self-Reported-Faller group (4).Participants are multipally labelled, which provides flexibility in use.However, it is up to the individual researcher to decide how to best use these labels.Thresholds of clinical-impairment.This section details the thresholds applied to each clinical test.
SPPB.The SPPB test was carried out using the protocol described in 27 and scored based on the scoring system shown in Table 6.The participant was classified using the criteria detailed in Table 7.Those who were classified as having Moderate Limitations or Severe Limitations were marked as at-risk for this test.
3 m walk.Quach et al. 28 used a 3 m walk to assess falls-risk in an 18-month-long, longitudinal study of 764 community-dwelling older people, mean age: 78 ± 5 years.They concluded that a walking speed of <0.6 m/s was associated with an increased risk of falling inside.This is equivalent to a time of >5 seconds to complete.Applying this threshold to the 3 m walks in KINECAL, anyone who took >5 seconds to complete the 3 m walk was marked as at-risk, for this test.

STS-5.
Another longitudinal study was undertaken by Ward et al. 26 .Over 4 years, they studied 755 community-dwelling older people, mean age: 78.1 ± 5.4 years.The study looked at SPPB as a predictor of injurious falls.The conclusion was that the STS-5 alone was enough to assess falls-risk.They suggested that a time to complete ≥16.7 seconds may be sufficient to identify those at risk of future falls.Applying this threshold to KINECAL, anyone who took >16.7 seconds to complete the STS-5 test was marked as at-risk, for this test.
TUG. Shumway-Cook et al. 25 found TUG to be a powerful test for fallers.They used a population of 30 people, split evenly between two groups, i.e.Non-Fallers (no historic falls in the last 6 months) with mean age: 78 ± 6 years, and fallers (≥2 falls in the last 6 months by history) with mean age: 86.2 ± 6 years.They suggested that those who take >14 seconds to complete the TUG are at elevated risk of falls.The Mc Kinly Laboratory 29 provides a reference for normative scores for TUG, synthesised from the Shumway-Cook paper and 3 more 24,30,31 .This reference extends the recommendations of Shumway-Cook et al. 25 , as presented in Table 8.In spite of the other work, the value presented in 25 remains the current recommendation to identify those at elevated risk of falls.Applying this threshold to KINECAL, anyone who took >14 seconds to complete the STS-5 test was marked as at-risk for this test.
Filtering.As with any system that digitises real-world data, Kinect's signal is bound up with noise, which can be seen as jittering of the skeletal joints.To address this issue, the recordings were filtered using a Butterworth fourth-order zero-lag filter with a cut-off frequency of 8 Hz.The cut-off frequency was calculated using residual analysis of the filtered and unfiltered signal as a function of the filter cut-off frequency, as described by Winter 32 .8 Hz provided the best balance between the signal distortion and the amount of noise allowed through.
Pose normalisation.Each skeleton frame was aligned to the first frame of the recording, making all subsequent movements relative to this initial position 33 , using Eq. 1. represents the position of the SPINE_BASE joint in the first frame of the recording, and P n,i (x, y, z) represents the positions of joint i in frame n.Estimation of CoM.In posturography, postural sway relates to the change in the position of the body's CoM over time.The most commonly used device for posturography is a force plate.When using force plates, CoM position is estimated from the Centre of Pressure (CoP) at the surface of a force plate 34 .Using the inverted-pendulum model, the position of the CoM is related to the position of the CoP by applying an offset related to an individual's height 35 .In our work, we derive CoM using a method described in our previous work 36 .It calculates CoM to be the 3D Euclidean mean of 3 joints of the Kinect skeleton: Hip left, Hip right, and Spine mid, as described in Eq. 2. This method has been validated to be equivalent to the CoM derived from the Balance Master in a previous paper 36 .

STS-5
Unable to complete or > 1 minute 0  Generation of sway metrics.The 3D CoM time series is a useful asset for researchers, as is the full depth and skeleton data.However, a set of common force plate metrics has been included for all upright stances.In doing this, the dataset simulates the type of lab-based metrics you would obtain from a force plate.Hence in one dataset, you have clinical and lab-based tests for each participant.Generally, when working with sway metrics derived from force plates, the CoM position is expressed in only two dimensions, i.e. the anatomical directions Anterior-Posterior (AP) and Mediolateral (ML).Kinect recordings are 3-dimensional, using a right-handed worldview.The origin is the centre of the camera, as shown in Fig. 4. To align the two views, the recordings were first normalised, and then the z-axis of Kinect was mapped to movement in the AP direction of the person being recorded.Similarly, the x-axis was mapped to movement in the ML direction.For this application, the y-axis was ignored.
Table 9 shows the abbreviations used for each metric, alongside a description of each metric.The abbreviations follow the form laid out in Prieto et al. 10 .

Sway metric -time series.
Force plates often have marks on their surface proscribing foot positions or will automatically centre the recordings.In the case of the recordings presented here.Centring was achieved by subtracting the mean CoM position, for an entire recording, from the CoM value at each time step.This was done separately for movement in the AP and ML directions using Eq. 4. The mean position was calculated using Eq. 3.
The output of Eq. 4 were concatenated to produce the AP and ML time series.A third time series was calculated, which takes into account movements in the AP and ML directions in a single value.Known as the resultant distance time series (RD), it was calculated as the vector distance from the mean CoM position to a pair of points in the AP and ML time series, at each time step.Values for each time step were generated using Eq. 5 and then concatenated to produce RD.Each of the calculated sway metrics was calculated using all three time series.

AP AP raw AP raw ML ML raw ML raw
If the AP is graphed against ML, the CoM path for each recording is revealed.Figure 5 compares the CoM paths of members of the Healthy-Adult and Clinically-Impaired groups Standing Quietly with eyes open (blue line).The 95% confidence ellipse is also shown (red ellipse) The calculation of the 95% confidence ellipse is detailed below.
Calculation of sway metrics.The following section gives details of the equations used to calculate the sway metrics.In these equations, _AP relates to the AP time series (AP).For the most part, these equations were also used to calculate the metrics in the ML and RD directions.The exceptions being MFREQ and the 95% confidence ellipse.where AP is the AP time series, i is a single time step and n is the total number of times steps, in the time series.ML was used in place of AP, to calculate the MDIST_ML.RD was used in place of AP, to calculate the MDIST.

RMS distance.
The root mean squared process removes the sign and gives more prominence to values further away from the mean CoM position.Equation 7 was used to calculate RDIST_AP.
where AP is the AP time series, i is a single time step and n is the total number of times steps, in the time series.ML was used in place of AP, to calculate the RDIST_ML.RD was used in place of, AP to calculate the RDIST.

Total excursion (CoM Path Length).
Total excursion is calculated by summing the distance between successive time steps.This is also known as the CoM path length.Equation 8 was used to calculate TOTEX_AP.

TOTEX AP
where AP is the AP time series and i is a single time step.ML was used in place of AP to calculate the TOTEX_ ML.RD was used, in place of AP, to calculate the TOTEX.

Mean velocity.
Mean velocity is the total excursion divided by the time for the trial, in seconds.Equation 9was used to calculate the MVELO_AP.
where AP is the AP time series and t is the time for the trial in seconds.ML was used in place of AP to calculate the MVELO_ML.RD was used, in place of, AP to calculate the MVELO.
Mean angular frequency AP and ML.MFREQ_AP is the frequency, in Hz, of a sinusoidal oscillation with an average value of the mean MDIST_AP and a total path length of TOTEX_AP.This was calculated using Eq.10.
where MVELO_AP is defined by Eq. 9 and MDIST_AP is defined by the Eq. 6. MVELO_ML was used in place of MVELO_AP and MDIST_ML was used in place of MDIST_AP to calculate the MFREQ_ML.

Mean angular frequency.
MFREQ is the mean angular frequency, in Hz.This is the number of revolutions per second of the CoM if it had travelled the total excursion around a circle with a radius of the mean distance.It is calculated using values derived from RD, given by Eq. 11.
where MVELO is defined by Eq. 9 and MDIST is defined by Eq. 6.
95% confidence elliptical area.The AREA_CE is given by the set of equations below (12).The elliptical area is an estimate of the area described by the maximum and minimum AP and ML values of the time series.To reduce the effect of rapid changes in direction, this value is scaled to 1.96 standard deviations of the mean value., where p is the Pearson correlation coefficient, ML r is the ML elliptical radius scaled to the 95% Confidence Interval (CI), and AP r is the AP elliptical radius scaled to the 95% CI.

Data Records
The raw data files and the calculated sway metrics are stored in a physionet repository 37 .The section below provides details of the structure of the data found there.Fig. 7 Box and whisker plots for STS-5 movement of the KINECAL and K3Da datasets: The range of movement in each joint is shown for the sagittal and frontal planes.similar ranges are seen between the two datasets.Independent t-tests were used to demonstrate no significant difference between the two datasets, shown in Table 10.

technical Validation
The recordings were undertaken as laid out in the Data Collection section.After recording, a video was created from the movements of the extracted joints, over time.Each frame represents the 3D coordinates in space for one time-step.the videos were then viewed to ensure all recordings are free of artefacts.Any videos with artefacts were excluded.
KINECAL represents a unique dataset, and as such, it is difficult to compare to other datasets.However, the previously published K3Da dataset does share similar movements.Using this dataset as ground truth, we compared the range of movement (ROM) for STS-5, in both the sagittal and frontal planes.In the comparison, we examined the healthy adult group in KINECAL and the participant's ages < 65 years old in K3Da.The results are shown in Fig. 7 and Table 10.The KINECAL data compares very well to the K3Da data.

Usage Notes
Both raw depth and joint positions are provided.These files can be processed using many commonly used data processing tools, e.g.Python and Matlab.
The data can be used to study both postural sway metrics, akin to data derived from force plates and whole body metrics, such as joint angles, akin to metrics derived from marker-based motion capture, such as VICON.Sample code is available 37 to demonstrate how this is achieved.

Fig. 1
Fig.1The relationship between an individual being recorded and the Microsoft Kinect camera.The CoM is calculated as the Euclidean average of the HIP_LEFT, HIP_RIGHT and SPINE_MID joints.

Fig. 2
Fig. 2 Kinect point-of-view This figure is of a young participant in quiet stance.(a) is a depth image, (b) is the 25-joint skeleton created by Kinect.
HIP_RIGHT, SPINE_MID are Kinect joints, indicated on Figure 3.The Figure also shows the resultant position of the CoM for a single frame.

Fig. 3
Fig. 3 Kinect V2 Skeleton: the joints used to estimate the CoM are labelled, along with the estimate CoM position.

Fig. 4
Fig.4 Kinect coordinate system: z-axis projects from the depth camera, with x and y orthogonal to it.This image is recreated from here.

Fig. 5 95%
Fig. 5 95% Confidence Ellipse and CoM path: This figure compares CoM paths and associated 95% Confidence Ellipses of a person labelled Healthy-Adult (25) to someone labelled Clinically-Impaired (201), Standing Quietly.Column (a) shows a tight view.Column (b) shows the 95% Confidence Ellipses plotted on the same scale.Note the majority of the increase in the area, for the Clinically-Impaired individual, come from movement in the AP direction.

Table 1 .
A comparison of KINECAL and previous datasets that contain clinically significant movements.

Quiet standing, Eyes closed, Firm surface Stand
Stand feet close together, eyes open and arms by your side.The test is terminated after 20 seconds, or if lost balance.feet close together, eyes closed and arms by your side.The test is terminated after 20 seconds, or if lost balance.

Table 3 .
Grouping of Self-reported Labels: This table shows the split between the different groups in terms of mean age, total numbers and gender split.A description of each group is also included.

Table 4 .
Grouped Sub-labelling of Self-reported-Fallers: This table shows the split between the different single (_s) and multiple fallers (_m), in terms of mean age, total numbers and gender split.A description of each group is also included.

Table 5 .
Details of the Clinically-At-Risk group: mean age, total numbers and gender split are shown.

Table 9 .
Table of sway metrics: This table details the sway metrics from the KINECAL dataset.Note.RD refers to the Resultant Distance, AP refers to Anterior-posterior, and ML refers to Mediolateral directions, details of how these metrics were calculated can be found in section 3.

Table 10 .
Comparison of the STS-5 movement from KINECAL and K3Da: The table shows a comparison of KINECAL and K3Da data.the p-value shown is from an independent t-test, and in all cases indicates no significant difference between the range of movement recorded by both datasets.