Introduction

Participating in sports and physical activities has numerous benefits for disabled individuals. Apart from improving the physical capacities, it helps reduce depression, improves family and social interaction and prolongs life expectancy.1, 2 Sir Guttmann believed that sport is a pathway that might help even severely disabled people to live a healthier and happier life, to gain confidence and self-esteem, and to achieve a degree of independence.3

Today, the paralympics are elite sport events for athletes from six different disability groups that emphasize the participant's athletic achievements rather than their disability. The paralympics have raised the status of disabled sport to the point where participants earn esteem as athletes in their own right, thereby challenging prevailing assumptions and stereotypes about ‘disability’.

Winning or losing an event should depend on training, talent, motivation and skills, rather than on belonging to a favoured or disadvantaged group.4 A functional classification system to minimize the influence of impairments on sport outcome is therefore of great importance. The International Paralympic Committee (IPC), the international governing body of sports for disabled athletes, defines functional classification as follows: ‘The categorization of competitors into classes on the basis of their performance potential, based on the relationship between impairment and sports activity’.5 Therefore, the classification criteria should be based on the relationship between the functional potential of the athlete and the determinants of a sport-specific performance.

Nordic skiing competitions are open to athletes with a physical disability (sit-ski and standing classes) and visually impaired athletes. It involves two disciplines: cross-country and biathlon. The IPC makes use of the ‘percentage system’ in which all disabled skiers compete against each other in three combined medal classes, namely ‘visually impaired’, ‘locomotor standing’ and ‘sitting’ classes. The system is an adjusted formula that is used to determine the overall score for each competitor relative to all other disabled racers.6

This study focuses mainly on the classification of the sit-ski classes for Nordic skiing, encompassing five subclasses: LW10, LW10.5, LW11, LW11.5and LW12.7 The criteria for these sitting classes are based on medical documentation of the athletes, including muscle tests, and functional tests to assess sitting ability and trunk stability. In spinal cord injury (SCI) the injury level is assessed using the American Spinal Injury Association (ASIA) classification.8 For the functional testing the test–table–test (TTT) was already introduced in 19859 and was adapted later by IPC classifiers.10 The TTT is a functional test testing sitting ability and trunk stability. During the TTT the participant is strapped on a stable board with supporting cushions under the knees and feet (see Figure 1a). The participant is asked to accomplish four tasks in which movements of 45° flexion, 45° backward inclination, lifting a ball above the head and maximum trunk rotation are required. Together with the medical documentation and the ASIA score (in case of SCI), the TTT result indicates a classification in one of five sitting classes.

Figure 1
figure 1

Cushion-padded seating board used during all tests (a) and overview of the ‘gold standard’ set-up (b). Note: Part of the safety padding is removed for pictorial clarity in (b).

Full size image

However, classification in disability sport is not evidence based, and objections and protests of both athletes and coaches occur against class allocation.

The aim of this study was to assess the interrater reliability and validity of the TTT with which Paralympic sports participants involved in Nordic sit-ski sports may be classified according to their level of physical ability related to sport. The research questions were as follows: Is the TTT reliable to classify Paralympic sports participants in Nordic sit-skiing? and Is the TTT valid to classify Paralympic sports participants in Nordic sit-skiing?

Materials and methods

The design of the study was cross-sectional.

Subjects

Persons with either a complete or an incomplete SCI at a level caudally to Th2, with a unilateral or bilateral leg amputation, with poliomyelitis affecting the trunk and/or lower extremities, or with spasticity due to cerebral palsy were asked to participate. Their age should have been between 18 and 70 years. Severe secondary pathology that might impede performance, such as severe cardiovascular impairments or pressure ulcers within 6 months before testing, was considered an exclusion criterion. All subjects should have completed their active rehabilitation program at least within 1 year. The participants did not necessarily have to be top athletes as the TTT is aimed at identifying the level of impairment rather than level of trained performance.

Eligible participants were identified using the databases of the Departments of Spinal Cord Injury and Amputation, Traumatology & Orthopaedics at Adelante Rehabilitation Centre in Hoensbroek, The Netherlands. Additionally, potential participants were contacted through various Dutch patient focus groups. Background information about injury level and severity, additional injuries and complications were collected by reviewing medical records. We certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during the course of this research. The study was approved by the Medical Ethics Committee of the Maastricht University. All participants gave their written informed consent before participation.

Tasks and apparatus

Classification and TTT. The classification procedure consisted of performing an ASIA impairment classification (AIS) in SCI participants through medical examination. Also, the so-called TTT, which is presently used in paralympics classification of Nordic sit-ski participants in categories LW10–LW12,7 was administered. The end result of the classification procedure is a single score indicating the class (one out of five) each participant is classified in.

During the TTT four physical tests were performed, ratings of which are presented in Table 1. The extent to which sitting balance could stably be maintained was determined by identifying the person's maximum reaching distance and the use of trunk muscles and compensation techniques (see also Table 1) observed by the classifiers during testing.

Table 1 Grading of the functional assessment on the test-table board
Full size table

Test 1: The participant sat with his/her hands behind the neck. He/she was asked to forward flex the trunk at the waist as much as possible, then extend the trunk and move to a position of 45° forward flexion indicated by a landmark. The position had to be maintained for 5 s while keeping the hands behind the neck.

Test 2: The seated subject was asked to fold the arms over the chest, lean back and maintain a 45° backward inclination of the trunk relative to the horizontal for 5 s. Subsequently, the subject was asked to return to the starting, complete upright, sitting position.

Test 3: The subject was asked to perform a maximum rotation of the trunk in the long-sitting position in both directions while keeping the arms fully abducted.

Test 4: The subject was asked to bimanually lift a 1-kg medicine ball over the head from the left to the right and back. Leaning on the ball had to be avoided.

Participants sat on a test board (see Figure 1a) consisting of a medium density fibreboard (MDF) padded with specially designed standardized cushions also supporting the legs. The position of these cushions could be adapted to the person's anthropometrics. Velcro straps over the hip joints, knees and ankles were used to secure the legs during classification testing.

Interrater reliability of the TTT was assessed by having two certified IPC classifiers (DP and AL), each rating each subject participating in the study independently, that is, blinded for each other's rating and in random order of appearance of participants.

Procedure ‘gold standard’ platform test. The validity of the TTT was assessed by comparing TTT results with a ‘gold standard’, that is, (simultaneously recorded) force plate recordings (Biovec-1000, AMTI, Watertown, MA, USA) during systematic sitting balance perturbation, analogous to the work by Seelen et al.11, 12, 13, 14, 15, 16 A test board was mounted on top of the force plate. Sample rate was 200 Hz. Sample time was individually adjusted for each participant to fully complete the activity required. The following movements were performed:

  1. 1)

    Reaching forward with both arms stretched out in sagittal direction.

  2. 2)

    Reaching 45° forward with the left arm stretched out and the right hand positioned on the chest.

  3. 3)

    Reaching 45° forward with the right arm stretched out and the left arm positioned on the chest.

  4. 4)

    Reaching lateral to the left side with the left arm in 90° flexion in shoulder and elbow and the right hand positioned on the chest.

  5. 5)

    Reaching lateral to the right side with the right arm in 90° flexion in shoulder and elbow and the left hand positioned on the chest.

Participants were asked to reach as far as possible without losing balance. The test board's Velcro straps were not used during the ‘gold standard’ testing. An overview of the ‘gold standard’ test set-up is presented in Figure 1b.

The movements required during the gold standard tests differed to some extent from those used in the TTT conditions, since the latter tests, involving submaximal trunk flexion or trunk rotation, led to small, submaximal and poorly reproducible centre of pressure (CoP) displacements. Yet, the TTT conditions were very useful in quickly assessing both postural balance control and the use of main (trunk and pelvis) muscle groups.

Data analysis

Force plate signals recorded were analysed using MATLAB software (The Math Works Inc., Natick, MA, USA). Maximal CoP displacement in all directions was calculated. Fenety et al.17 have shown the linear relationship between the position of CoP and the angles of trunk inclination and lateral flexion. Validity of the TTT was statistically assessed by correlating TTT ratings with the maximal CoP displacements. As for interrater reliability, statistical analysis included the calculation of Spearman's rank-order correlation coefficients.18

Statistical analyses were performed using SPSS software (SPSS Inc., Chicago, IL, USA).

Results

Participants

Thirty-three persons participated in the study. Group composition is presented in Table 2.

Table 2 Group composition
Full size table

TTT classification and interrater reliability

TTT classifications for all participants per TTT subtest by both classifiers are presented in Table 3.

Table 3 Overview of the classification of subjects
Full size table

As for the interrater reliability regarding the classification of subjects, the Spearman's rank-correlation coefficient was 0.95 (P<0.001). Interrater reliability data did not differ as a function of rating level, that is, any disagreement between raters was not typically prevalent in, for example, high rater scores or low rater scores.

Validity

An example of CoP displacement while reaching the lateral direction of one of the participants is presented in Figure 2.

Figure 2
figure 2

Example of CoP displacement during reaching in lateral (left) direction of one of the participants. Cross (0.00,0.00)=baseline position; Dot (−0.10,0.02)=maximal CoP displacement.

Full size image

Boxplots describing CoP results per TTT subclass for the anterior (pooled left and right), lateral and (pooled left and right) diagonal reaching directions are presented in Figures 3, 4 and 5.

Figure 3
figure 3

Boxplots of anterior CoP displacements per test–table–test subclass.

Full size image
Figure 4
figure 4

Boxplots of lateral CoP displacements per test–table–test subclass.

Full size image
Figure 5
figure 5

Boxplots of diagonal CoP displacements per test–table–test subclass.

Full size image

Correlation coefficients ranged from 0.74 (anterior reaching condition) to 0.61 (lateral reaching condition) and 0.70 (diagonal reaching condition) (P<0.001).

Discussion

The aim of this study was to assess the interrater reliability and validity of the TTT with which paralympic Nordic sit-ski participants may be classified according to their level of physical ability related to the sport. The interrater reliability data showed high levels of agreement in both scoring and classification. As for TTT validity, strong positive correlations between CoP displacement and TTT classification were found, although in classes LW10 and LW10.5 the correlation is less clear.

The TTT is one part of an extensive classification procedure in sit-skiing sports. Next to the TTT, medical documentation and the ASIA classification (in case of SCI), actual performance on the track outside is assessed in each athlete. During the latter, special attention is paid to changing of the tracks using trunk and hip assistance; trunk assistance during climbing, trunk stability and control during hill descent; and trunk control in curves. All these results are evaluated by the sport technical and medical classifiers (the classification team) before the final classification is determined. However, despite the ongoing development and refining of the classification systems for disability sports, no scientific evidence for the use of the current classification system in Nordic sit-skiing was available. This lack triggered the set-up of the current study.

Fair classification in sports for the disabled involves not only a fair ranking/scoring system but also an unambiguous judgement by the classifier(s) involved. Therefore, the interrater reliability of the TTT classification was assessed and, despite the overall good interrater reliability, in 4 out of 33 participants disagreement was still present, indicating that further refinement is still necessary. Currently, the two classifiers who participated in the study are the most experienced Nordic sit-ski classifiers in the world and are well acquainted with each other's way of testing. During the training of additional classifiers special attention should be paid to the interrater reliability issue and the further standardisation of protocols used.

By comparing the results of the gold standard test with the currently used TTT results, the validity of this latter test was studied. CoP displacements were taken as a measure to determine the ability to maintain both equilibrium and posture counteracting perturbing internal and external influences.16 In persons with a thoracic SCI it was shown that the domain in which the CoP can be actively controlled is reduced relative to the CoP domain in non-SCI subjects.13, 14 As CoP displacement (gold standard) can be seen as an indirect measure of sitting ability, a positive correlation between the functional sitting ability and the TTT classification was expected. This study, in general, though not fully, corroborated this expectation. For example, the relations between TTT and CoP displacement in the LW10 and LW10.5 subclasses found in this study are more vague. The latter may be due to the small number of participants in these subclasses. For the LW10 and LW10.5 subclasses further refinement of the four tests within the TTT may be warranted. Furthermore, it should be kept in mind that the TTT is only a part of the complete classification, as was mentioned earlier, which might explain why the correlations found between CoP data and participants’ classification were not even higher. We did not test the participants in a sit-ski, which is individually designed and adapted to the individual, so we do not know the effect of the equipment on the functional performance. Further research is needed in this area.

Possible limitations of the study

Several factors, such as body length, age or co-morbidity, might possibly have influenced either the TTT results or the CoP displacement results. For example, Boswell-Ruys et al.19 indicated that subjects with a longer trunk perform more poorly on maximal balance tests than subjects with a smaller stature. However, after having normalized CoP displacement data for individual body length, results did not change significantly. Obviously, body length dispersion among TTT subclasses was quite even. Alternatively, body length could have influenced both CoP data and TTT classification to the same extent, although this seems somewhat unlikely, given the different scoring systems/scales used. As for age, Thompson and Medley20 described that the sitting balance of older participants differed from that of younger participants during forward and lateral reaching tasks. However, in the current study no relation between age and CoP displacement was found. Moreover, some older participants performed very well on both tests in comparison with younger participants. As for co-morbidity, some variety was indeed found in the group of participants, but no relation with the TTT classification was found. In comparison with the professional sit-ski population, age and physical condition may vary more between our participants. Professional Nordic sit-ski competitors are younger compared with the study group and have a better physical condition. However, the TTT is aimed at identifying level of impairment rather than level of trained performance, making it very unlikely that (trained) physical condition could have obscured the results.

Future research

Balance and sitting ability are important not only in sit-skiing but also in a wide variety of other sports such as wheelchair tennis, wheelchair table tennis, wheelchair rugby and basketball, wheelchair hockey, equestrian and rowing. These sports might also benefit from classification methods based on adapted TTT in the future, which could lead to further improvement in fair classification in sports for the disabled.