Abstract
Study design:
Psychometric study
Objectives:
To develop and test a valid and reliable assessment of wheelchair skills for individuals with spinal cord injuries (SCI); the Queensland Evaluation of Wheelchair Skills (QEWS).
Setting:
Hospital, Australia.
Methods:
Phase 1: Four Delphi panel rounds with clinical experts were used to develop the QEWS. Phase 2: Intra-rater and inter-rater reliability of the QEWS items were examined in 100 people with SCI. Phase 3a: Concurrent validity was investigated by examining the association between QEWS total scores and physiotherapists’ global ratings of wheelchair skill performance. Phase 3b: Construct validity was tested in 20 people with recent SCI by examining change in QEWS total scores between when they first mobilised in a wheelchair and scores obtained 10 weeks later.
Results:
Phase 1: The QEWS was developed. Phase 2: The intra-class correlation coefficients reflecting the intra-rater reliability and the inter-rater reliability for the QEWS total score were 1.00 and 0.98, with scores being within one point of each other 96 and 91% of the time, respectively. Phase 3a: The QEWS total scores were comparable with the global rating of wheelchair skill performance (r2=0.93). Phase 3b: The QEWS scores changed by a median (interquartile range (IQR)) of 4 (1 to 6) points over the 10-week period following first wheelchair mobilisation.
Conclusion:
The QEWS is a valid and reliable tool for measuring wheelchair skills in individuals with SCI. The QEWS is efficient and practical to administer and does not require specialised equipment.
Similar content being viewed by others
Introduction
There are currently several ways to assess the wheelchair skills of people with spinal cord injury (SCI). However, none of the methods available are ideal for the typical hospital setting while also being appropriate for the community.1 Existing assessments of wheelchair skills include; the Wheelchair Skills Test 4.2, Wheelchair Circuit Test, Adapted Wheelchair Circuit Test, Obstacle Couse Assessment of Wheelchair User Performance Test, The Five Additional Mobility and the Locomotor Item Test (which is used in conjunction with the Functional Independence Measure) and the Test of Wheeled Mobility and Short Wheelie Test.2, 3, 4, 5, 6, 7 Each of these assessments have their strengths, however none of the existing assessments are quick to administer while providing a summary of a person’s wheelchair skills that is sensitive to the types of changes seen in people with acute SCI. For example, the Wheelchair Skills Test 4.2, Obstacle Course Assessment of Wheelchair User Performance Test and the Test of Wheeled Mobility and Short Wheelie Test all take at least 30 min to complete.3, 5, 7 The Wheelchair Skills Test 4.2 was designed as a comprehensive checklist to guide wheelchair training programs and is appropriate for this purpose, but it is not a quick assessment that can be used in routine clinical practice to monitor change over time.8 In addition, most existing wheelchair skill tests cannot be easily performed in the typical hospital setting without establishing a dedicated environment that includes access to relatively large open spaces and specialised equipment. Specifically, the Wheelchair Circuit Test requires an adapted treadmill for wheelchairs, and the Test of Wheeled Mobility and Short Wheelie Test require six ramps of varying gradients and four different gutter thresholds.3, 4 The Obstacle Course Assessment of Wheelchair User Performance Test is also difficult to replicate because it requires access to a gravel pathway and a variety of gutters and ramps.7
Another problem with existing assessments is their inclusion of items that are not directly related to wheelchair skills. Unrelated items include those related to either transferring or moving about on a bed, and items related to other skills, such as folding up a wheelchair or opening a door.1, 2, 3, 4, 5, 6, 7 Most of these types of skills are already captured in the Spinal Cord Independence Measure (SCIM) or other equivalent outcome measures.9 Therefore, it is not efficient to duplicate these measures in clinical practice when assessing wheelchair skills. Accordingly, there is a need for a quick and simple test that can be used in conjunction with the SCIM or equivalent measures, and that focuses exclusively on a small set of key wheelchair skills. Therefore the purpose of this study was to develop and examine the reliability and validity of a test constructed to objectively assess the skills of manual wheelchair users with SCI. The test was designed for use in the acute hospital setting as well as in the community without requiring access to extensive or specialised testing equipment. It reflected skills necessary for both indoor and outdoor manual wheelchair use.
Methods
Phase 1: development of the tool
A literature review was conducted and followed by a Delphi panel process to develop the tool. The panel consisted of 12 physiotherapists with extensive clinical expertise across the continuum of care of SCI rehabilitation. Members of the research team facilitated the Delphi panel process (EJG, SMM). During the first Delphi round, the panel identified 31 items from existing wheelchair skill assessments for potential inclusion. In the second Delphi round, panel members were required to comment on whether items reflected competency in wheelchair skills and whether items were of relevance to people with SCI. They were also asked about the feasibility of scoring items in the hospital setting. From this, a prioritised list was determined. To be eligible for inclusion, items needed to capture changes in wheelchair skills across the continuum of care. They also needed to be testable within the acute hospital setting without requiring costly equipment or excessive space. The third Delphi round was held to achieve consensus regarding the final 5 items for inclusion in the instrument, with each item examining a key attribute of wheelchair mobility. The panel members provided confidential feedback to the research team throughout the process either in person or by telephone or email. The feedback was summarised anonymously and distributed in written format to the panel members. Consumer feedback from 2 individuals with acute SCI undergoing primary rehabilitation and 2 individuals living in the community with a SCI was sought. Feedback was requested on each of the selected items. During the fourth and final Delphi round, consensus was reached regarding the scoring needed for each item to ensure it had enough range to capture the varying levels of ability and competency of individuals with SCI who self propel a manual wheelchair. The resulting assessment tool was called the Queensland Evaluation of Wheelchair Skills (QEWS) (Refer to Appendix 1).
Phase 2: reliability study
The intra-rater and inter-rater reliability of the QEWS were determined by testing 100 participants who were recruited in two ways: either on admission to the Princess Alexandra Hospital Australia following an acute SCI, or through outpatient departments and local community organisations via the use of information brochures and posters displayed in waiting rooms. Participants were included if they had sustained a SCI and used a manual wheelchair as their primary form of mobility. Individuals were excluded if they were unable to propel a manual wheelchair independently indoors over flat surfaces or were unable to follow basic instructions. Informed consent was obtained from all participants and all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during the course of this research.
Participants were scored three times; once by a first assessor and then twice by a second assessor. The first assessment was done in person. The participants’ performance was video recorded at this time. Using the video recording of the first assessment, a different assessor did the second and third assessments. There was a period of at least 2 weeks between the second and third assessments (Refer to Figure 1). All assessments were completed and scored by qualified physiotherapists. A total of ten physiotherapists were used across the course of the study, and they had a median (IQR) of 15 (9 to 26) years clinical experience in SCI (Refer to Table 1). Their assessment scores were used to determine both the intra-rater reliability and inter-rater reliability
Intra-rater reliability was determined by comparing scores completed by the same rater on two occasions, and inter-rater reliability was determined by comparing scores from the two different raters of the same performance. For both sets of analyses, Cohen’s weighted kappa coefficient was used to evaluate reliability for each of the five items of the QEWS, with linear weighting applied to penalize disagreements of more than one level on ordinal item scales. An intra-class correlation coefficient (ICC) was used to determine reliability for the QEWS total scores. In addition, percent close agreements were calculated and Cronbach’s alpha was used to examine the internal consistency of the five items of the QEWS.
Phase 3: validity study
The third phase of the study was to assess the validity of the QEWS. This was done in the following two ways.
Phase 3a (concurrent validity): This was tested in the 100 participants used for Phase 2. The QEWS total score of the 100 participants was compared with physiotherapists’ global ratings of wheelchair skills, which were completed on the same days as the QEWS. The global rating involved rating overall performance on an 11-point scale that was anchored at the ends by the labels ‘very poor performance’ and ‘very high performance’. Generalized linear modelling was used to examine the association between the QEWS total scores and the physiotherapists’ global ratings of wheelchair skill performance. A model fitting exercise was conducted to determine the most appropriate model parameters for the data, using the Akaike Information Criterion as an indicator of model fit with penalty for model complexity. Gaussian (family) and Identify (link) parameters produced the closest model fit, although the significance of the association was consistent regardless of parameter selection. This association was also represented on a scatterplot with a line of best fit (Refer to Figure 2).
Phase 3b (construct validity): This was tested in a subgroup of twenty participants who had recently sustained a SCI and commenced rehabilitation. They were assessed using the QEWS within 2 weeks of commencing rehabilitation and then 10 weeks later. The median (and IQR) change in QEWS total scores between the first and second assessment was calculated. These data were used to give an indication of the responsiveness of the instrument to improvement in wheelchair skills over time.
Results
Phase 1: Development of the QEWS
The QEWS consists of five items. Each item is scored on a 6-point scale ranging from 0 to 5 with a score of 0 indicating poor performance and a score of 5 indicating good performance. The scores for each item are tallied to a total possible score of 25. A brief description of each item is as follows (Refer to Appendix 1 for details).
Item 1: negotiating an indoor circuit
This involves 5 components, namely, propelling forwards, moving over a piece of wood (to represent a doorway threshold), negotiating cones in a figure of eight pattern, turning 180 degrees and propelling backwards. The number of components that the individual can independently perform determines the score.
Item 2: ascending and descending a ramp
This item examines the ability of an individual to ascend and descend a ramp. A 1:14 gradient ramp is required to reflect the ramps commonly used to access homes and community buildings. The score is determined by a combination of assistance required and time taken to ascend and descend the ramp.
Item 3: maintaining balance on back wheels
This item reflects the ability of an individual to maintain balance on the back wheels in a variety of positions; stationary position, moving forwards, turning 90 degrees, turning 360 degrees and descending a 1:14 gradient ramp. The score is determined by the difficulty of the skill performed.
Item 4: ascending and descending a gutter
This item consists of two parts. Firstly, the ability to ascend a 15 cm gutter and secondly, to descend a 15 cm gutter. The 15 cm gutter was selected to represent the height of most community gutters. A hierarchical scoring system is used, with the score reflecting the complexity of the skill that the individual can independently perform.
Item 5: six-minute push test
This item was included to provide an indication of an individual’s fitness and ability to propel a manual wheelchair for an extended period of time.10 The item involves pushing as far as possible in six minutes, and the distance pushed determines the score.
All scoring is done by observing the participant attempting the item. A score of 0 is given if the therapist or participant deems it unsafe to attempt an item.
Phase 2: Reliability Study
The demographic information for participants in the study is provided in Table 2. The results for the intra-rater and inter-rater reliability are shown in Table 3. The weighted Kappa values for each of the five items of the QEWS, for the intra-rater and inter-rater designs, ranged between 0.84 and 0.99, indicating very good reliability. In terms of agreement, scores were within one point of each other between 97 and 100% of the time. The intra-class correlation coefficients (and associated 95% confidence intervals) for the QEWS total score reflecting the intra-rater reliability and inter-rater reliability were 1.00 (0.99 to 1.00) and 0.98 (0.98 to 0.99), respectively, indicating good to perfect reliability.11 These scores agreed within one point of each other 96 to 91% of the time. The internal consistency reliability coefficient (Cronbach’s alpha) of the five items was 0.86.
Phase 3: Validity study
Phase 3a (concurrent validity): A strong linear association (r2=0.93) between the QEWS total score and the concurrent global rating of wheelchair skills is displayed in Figure 2. The generalised linear modelling confirmed this to be a significant association (coefficient=2.30, 95% confidence interval=2.18 to 2.43, P<0.001).
Phase 3b (construct validity): The QEWS total score was higher at the second assessment by a median (IQR) of 4 (1 to 6) points.
Discussion
The QEWS is a quick, reliable and valid assessment of wheelchair skills in people with SCI. The test does not require specialised equipment although it does require a stopwatch, cones and piece of wood (to represent a doorway threshold) as well as access to a 10 and 30 m track, a ramp and a gutter. Each assessment can be completed in less than 20 min and sometimes considerably less depending on the characteristics of the individual. Item 5 is the most time consuming aspect of the QEWS because it includes a six-minute push test.
The intra-rater and inter-rater reliability of the five items of the QEWS and the QEWS total score were found to be very good and sometimes perfect. The use of videos to test reliability had both benefits and drawbacks. On the one hand the videos may have over-stated reliability because they removed some of the inherent variability that is associated with real-life assessments due to small differences in set-up and instructions to participants. On the other hand, the videos sometimes made it more challenging to rate performance because of difficulties in determining the amount of physical assistance required, particularly for item 4 (ascending and descending a gutter). In addition, it was sometimes difficult to determine the time taken to complete item 2 (ascending and descending a ramp) from the video because the start and finish lines were hard to see, where as this was not an issue for the therapists during real life assessments.
The validity of the QEWS was reflected in the high correlation between the QEWS total scores and the physiotherapists’ global ratings of wheelchair skills. Similar to other studies, the global rating of wheelchair skills was used to reflect therapists’ overall impression of the participants’ wheelchair skills.8 However, our estimate of the strength of association between physiotherapist global rating and the QEWS needs verifying in future studies, because the same therapists scored both the QEWS and global ratings, and it would be a stronger test if the two scores were provided by different therapists. Validity was also examined by demonstrating changes in scores over time. There was a median (IQR) of a 4/25 (1 to 6) point change in the QEWS total scores over the 10-week period. This amount of change was anticipated and demonstrates that the QEWS is appropriately sensitive to change in wheelchair mobility skill over time. The amount of change over time was not influenced by level of lesion. The change in the QEWS total score for participants with paraplegia (median, 4; IQR, 2 to 6) was similar to participants with tetraplegia (median, 4; IQR, 1 to 6). There was good spread of data across the QEWS scale (see Figure 2). This indicates that the QEWS does not have floor or ceiling effects.
Another important aspect of this study is that it provides important normative data on the wheelchair skills of 100 people with SCI (Refer to Table 2). To our knowledge, no other study has previously provided this type of information for such a large sample. This information could be used by therapists to gauge the typical level of wheelchair mobility attained by people with different types of SCI.
Conclusion
The QEWS is quick to administer and easy to use in the acute hospital setting. It is also reliable, valid and sufficiently sensitive to detect change in wheelchair mobility skills over a 10-week period. Further work is required to determine the usefulness of the instrument in the community settings.
Data archiving
There were no data to deposit.
References
Fliess-Douer O, Vanlandewijck YC, Lubel Manor G, Van Der Woude LH . A systematic review of wheelchair skills tests for manual wheelchair users with a spinal cord injury: towards a standardized outcome measure. Clin Rehabil 2010; 24: 867–886.
Cowan RE, Nash MS, De Groot S, Van Der Woude LH . Adapted manual wheelchair circuit: test-retest reliability and discriminative validity in persons with spinal cord injury. Arch Phys Med Rehabil 2011; 92: 1270–1280.
Fliess-Douer O, Van Der Woude LH, Vanlandewijck YC . Reliability of the test of wheeled mobility (TOWM) and the short Wheelie test. Arch Phys Med Rehabil 2013; 94: 761–770.
Kilkens OJ, Dallmeijer AJ, De Witte LP, Van Der Woude LH, Post MW . The wheelchair circuit: Construct validity and responsiveness of a test to assess manual wheelchair mobility in persons with spinal cord injury. Arch Phys Med Rehabil 2004; 85: 424–431.
Kirby R. Wheelchair Skills Test (WST) Version 4.2 Manual Halifax, Nova Scotia, Canada: Dalhousie University; 2013 [cited 2015 07/01/15]; Available from http://www.wheelchairskillsprogram.ca/eng/testers.php.
Middleton JW, Harvey LA, Batty J, Cameron I, Quirk R, Winstanley J . Five additional mobility and locomotor items to improve responsiveness of the FIM in wheelchair-dependent individuals with spinal cord injury. Spinal Cord 2006; 44: 495–504.
Routhier F, Vincent C, Desrosiers J, Nadeau S, Guerette C . Development of an obstacle course assessment of wheelchair user performance (OCAWUP): A content validity study. Technol Disabil 2004; 16: 19–31.
Kirby RL, Swuste J, Dupuis DJ, MacLeod DA, Monroe R . The Wheelchair Skills Test: A pilot study of a new outcome measure. Arch Phys Med Rehabil 2002; 83: 10–18.
Catz A, Itzkovich M . Spinal Cord Independence Measure: Comprehensive ability rating scale for the spinal cord lesion patient. The Journal of Rehabilitation Research and Development 2007; 44: 65.
Cowan RE, Callahan MK, Nash MS . The 6-min push test is reliable and predicts low fitness in spinal cord injury. Med Sci Sports Exerc 2012; 44: 1993–2000.
Altman DG . Practical statistics for medical research, First ed. Chapman and Hall: London England, 1991.
Pulmonary Rehabilitation Tool Kit: The Australian Lung Fundation and The Australian Physiotherapy Association; 2009 [10/01/15]; Available from http://www.pulmonaryrehab.com.au/index.asp?page=19.
Acknowledgements
Special thanks are due to Roxanne Healy for assisting with getting this project started and Christine Dalais for support along the way. We would also like to thank all the therapists from the Queensland Spinal Cord Injuries Service and the participants who volunteered their time to assist with data collection. SMM is supported by an NHMRC Career Fellowship (Clinical Research).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no conflict of interest.
Appendix 1
Appendix 1
The Queensland Evaluation of Wheelchair Skills (QEWS)
General Principles:
-
Used to measure manual wheelchair skills for individuals with spinal cord injury.
-
Instructions are to be provided to the participant to ensure each item is performed safely.
-
A participant is not deemed independent if physical assistance, verbal cues for safety or supervision are required.
-
More than one attempt of each item is acceptable to accurately determine the skill level of the participant.
-
Only items deemed safe to be attempted should be assessed.
-
Assistive devices include grade aids, power assist wheels and use of a sign-post or rail.
Item 1: Negotiating an indoor circuit
-
10 m course marked out on gymnasium floor or equivalent (that is, indoor, flat and hard surface). Set up as per Figure 1a.
-
The score is determined by the number of components (listed below) that the participant can complete independently.
-
The 5 components are:
-
1
Propelling forwards from the start line
-
2
Continuing to propel forwards over the lip (dimension of piece of wood used is length 100 cm, width 4.5 cm and height 2 cm) which is placed 2.5 m from the start line
-
3
Continuing forwards to manoeuvre around three obstacles placed 1 m apart (that is, complete a figure of 8 around the cones)
-
4
Completing a 180 degree turn
-
5
Propelling backwards to the finish line
-
1
Rights and permissions
About this article
Cite this article
Gollan, E., Harvey, L., Simmons, J. et al. Development, reliability and validity of the queensland evaluation of wheelchair skills (QEWS). Spinal Cord 53, 743–749 (2015). https://doi.org/10.1038/sc.2015.82
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/sc.2015.82
This article is cited by
-
INTERnational Project for the Evaluation of “activE Rehabilitation” (inter-PEER) – a protocol for a prospective cohort study of community peer-based training programmes for people with spinal cord injury
BMC Neurology (2020)
-
Active Rehabilitation for persons with spinal cord injury in Botswana – effects of a community peer-based programme
Spinal Cord (2019)
-
Development of a wheelchair mobility skills test for children and adolescents: combining evidence with clinical expertise
BMC Pediatrics (2017)