Abstract
Three questionnaires are recommended in the management of chronic obstructive pulmonary disease by the global initiative for obstructive lung disease, of which two are the more comprehensive assessments: the chronic obstructive pulmonary disease assessment test and the clinical chronic obstructive pulmonary disease questionnaire. Both are carefully designed high-quality questionnaires, but information on the feasibility for routine use is scarce. The aim of this study was to compare the time to complete the chronic obstructive pulmonary disease assessment test and the clinical chronic obstructive pulmonary disease questionnaire and the acceptability of the questionnaires. Furthermore, the agreement between electronic and paper versions of the questionnaires was explored. The time to complete the electronic versions of the questionnaires was 99.6 [IQR 74; 157] vs. 97.5 [IQR 68; 136] seconds for clinical clinical chronic obstructive pulmonary disease questionnaire and chronic obstructive pulmonary disease assessment test, respectively. The difference in time to complete the questionnaire was not significant. The two questionnaires did not differ in “easiness to complete” or “importance of issues raised in questionnaires”. Electronic vs. paper versions revealed high agreement (ICC CCQ = 0.815 [0.712; 0.883] and ICC CAT = 0.751 [0.608; 0.847]) between the administration methods. Based on this study it can be concluded that both questionnaires are equally suitable for use in routine clinical practice, because they are both quick to complete and have a good acceptability by the patient. Agreement between electronic and paper versions of the questionnaires was high, so use of electronic versions is justified.
Similar content being viewed by others
Introduction
Questionnaires are recommended in the management of chronic obstructive pulmonary disease (COPD).1, 2 Over the last years several compact questionnaires addressing health status have been specifically designed to be used in routine clinical practice. The use of these questionnaires is thought to improve communication3 and can guide treatment.2, 4
Since 2011, the global initiative for obstructive lung disease (GOLD) guidelines/strategy5 has included three questionnaires in the assessment of COPD patients: the modified Medical Research Council (mMRC) dyspnea scale,6 the COPD assessment test (CAT),3 and the clinical COPD questionnaire (CCQ).7 As the mMRC solely addresses dyspnea, the CAT and CCQ are the more comprehensive assessments of the three questionnaires providing the clinician with more valuable information regarding burden of disease. For that reason we focus in this study on the head-to-head comparison between CAT and CCQ. The CAT consists of eight items scored on a 5-point scale and a total score can be calculated. The CCQ consists of 10 items scored on a 6-point scale, and a total score as well as symptom, functional, and mental status domain scores can be calculated.
Guideline developers and clinicians need to make choices on which of these two health status questionnaires they should recommend and use in daily practice. Next to the choices that have to be made on the content, quality, responsiveness, and comparability of these questionnaires, the feasibility for actual use in clinical practice is essential for successful implementation.
To date, only scarce information is available that compares the feasibility of the CAT and the CCQ. We studied both questionnaires regarding the average time to complete and acceptability. Furthermore, agreement between electronic and paper versions of both questionnaires was explored.
Results
COPD patients were invited to the study (Fig. 1). In total, 95 COPD patients participated in the study and completed the questionnaires online. Baseline characteristics are presented in Table 1. The total scores could not be calculated in 10% of CAT, and in 1% of the CCQ and mMRC due to missing values.
Differences between electronic version of the CAT and CCQ
The differences between the electronic version of the CAT and CCQ were tested in 95 patients and reported in Table 2. The median CAT completion time was 97.5 [IQR 68–136] vs. a median CCQ completion time of 99.6 [IQR 74–157] seconds (p = 0.151).
The easiness to complete, importance of issues raised in questionnaires, and the importance of information for health-care provider were not different between the two questionnaires.
Of the participants who completed the question on questionnaire preference (n = 78), the majority (79.5%) had no preference for one of the questionnaires, while 16.7% preferred the CCQ and 3.8% preferred the CAT.
Agreement between electronic and paper versions of the CAT and CCQ
For comparison of the electronic and paper versions, only stable patients (n = 65) were included in the analysis. Of those, 64 had no missing data on the electronic or paper version of the CCQ and 55 had no missing data on the CAT. The absolute agreement between electronic and paper versions was high (ICC [95% CI] CCQ=0.815 [0.712; 0.883] and ICC [95% CI] CAT=0.751 [0.608; 0.847]). Interpretation of Bland–Altman plots essentially supports agreement between the two versions (Figs 2, 3).
Discussion
Main findings
To the best of our knowledge, this is the first study comparing CCQ and CAT head-to-head in a primary care population to assess completion time and ease of use. This study demonstrates that in a primary care COPD population neither statistically nor clinically relevant differences in filling out time or acceptability between the two health status questionnaires were observed. Agreement between electronic and paper versions of the questionnaires was high.
Interpretation of findings in relation to previously published work
One earlier study by Ringbaek8 in a group of mainly severe COPD patients participating in a pulmonary rehabilitation program found patients needed 107 s to complete CAT, and 134 s to complete CCQ, but did not report whether this difference was significant. They also found that the need for assistance while answering the questionnaire was 53.9% for CAT, and 36.0% for CCQ. In our study, no assistance was given, but in the CAT questionnaire more questions remained unanswered than in the CCQ (10 vs. 1%). In contrast, Sundh et al. found a considerably higher proportion of the patients were able to complete the CAT compared with the CCQ,9 although they found that a slight majority of the patients preferred the CCQ.
In a study by Tsiligianni and colleagues,10 10% of the patients reported that the CCQ was easier to understand than the CAT. However, 62% indicated that the CCQ reflected their health status better than the CAT did because the CCQ addresses their breathing problems better while sleep was less important to them.10 In our study no differences were found regarding easiness to complete the questionnaires or the importance of the topics addressed between the questionnaires.
The agreement between electronic and paper versions of both CAT and CCQ was high, which justifies electronic use in daily practice or by patients themselves online.
The decision on which health status questionnaire to use can be made based on local preferences, or based on the fact that in addition to the total score the CCQ offers three subdomain scores (symptom, functional, and mental status) that can be used to guide treatment prioritization in practice. Furthermore, the usefulness of the questionnaires from a clinician’s perspective, including the ease of deriving a score, should be taken into account. Also the impact of missing items on the score or the impact of remotely completing the questionnaires should be considered. However, that was not within the scope of this study. This study adds to our knowledge the acceptance and feasibility for the patients in addition to the known feasibility for primary care as assessed by the IPCRG.11
Strengths and limitations of this study
A limitation of this study is that we could not rule out selection bias. Patients are invited to participate, and non-response or reasons for not participating are not examined. Moreover, characteristics between participants and non-participants could not be compared. Selection bias might reduce generalizability. Although it can be thought that factors like COPD severity, age, or comorbidity might influence time to complete the questionnaire, we expect that this will be the same for both questionnaires.
Importance of topics addressed is more likely to depend on severity, since more severe patients might have other or more symptoms than less severe patients. Our population is relatively mild based on mMRC, CAT, and CCQ scores. We cannot be sure whether the results of this study on importance of the topics addressed in the questionnaires can be generalized to more severe populations.
Another factor that might influence the results, especially easiness of the questions, is socio-economic status (SES). Participants are included from different parts of the Netherlands and from different general practices. Nevertheless, we cannot be sure that we included patients from with different SES levels.
Finally, the questions regarding acceptability of the CAT and CCQ were not pilot tested. Nevertheless, those questions were quite straight forward and the majority of the patients completed those questions. Because the same questions were used for both questionnaires we believe it is legitimate to draw conclusions based on those questions.
Implications for future research, policy, and practice
Based on this study, both questionnaires seem equally suitable for use in routine clinical practice, because they are both quick to complete and have a good acceptability by the patient.
Conclusions
No significant differences were found in the completion times and acceptability between the CCQ and the CAT in Dutch primary care. Based on this study it can be concluded that both questionnaires are equally suitable for use in routine clinical practice, because they are both quick to complete and have a good acceptability by the patient. Agreement between electronic and paper versions of the questionnaires was high, so use of electronic versions is justified.
Methods
Patients
COPD patients with a doctors’ diagnosis of COPD according to current guidelines12 were recruited from general practices and primary care rehabilitation physiotherapy programs in and around the Groningen and Rotterdam areas in the Netherlands. Inclusion criteria were (1) a doctor’s diagnosed COPD and (2) electronic informed consent. Exclusion criteria were (1) inability to understand or read the Dutch language and (2) unable to connect to the internet.
Primary care practitioners identified patients with COPD and invited them by letter or in person to participate in the study.
Participants logged in on the website to sign an electronic informed consent form and start with the study.
Outcomes
The primary outcome was the time (in seconds) required to complete the electronic versions of the CAT and CCQ questionnaire. The time running from showing the questionnaire on the screen until clicking “completed, next page” was recorded.
The CCQ is an instrument to measure health status in patients with COPD.7 It consists of 10 questions on three domains: symptoms, mental state, and functional state. The symptoms and functional state domains contain four items each, and the mental state domain two. Questions are scored on a 7-point scale from 0–6, with 0 representing the best possible score and 6 representing the worst possible score. Total scores range from 0–6. A higher CCQ value indicates a lower health status.
The CAT also measures health status in patients with COPD. The CAT has eight items and includes questions about symptoms, energy, sleep, and activity.3 Total scores range from 0 to 40, where 0 represents no impairment.
Secondary outcomes were the degree of acceptability, measured with three questions on easiness (How difficult was completing the questionnaire?), importance of issues raised in questionnaires (To what extent did you feel the questionnaire addressed all aspects of your disease?) and importance of information for health-care provider (To what degree do you think that the questionnaire gives relevant information about your disease to your doctor?), scored on a visual analog scales ranging from 0 to 10, where 0 indicates a low acceptability and 10 indicates high acceptability. In addition, patients were asked to indicate which questionnaire they would recommend to their general practitioner.
The MRC questionnaire is a one-dimensional tool to measure dyspnea during exercise in five levels (range 0–5).6 The global rating of change (GRC) questionnaire was used to assess change in breathlessness between completing the electronic and paper questionnaires on a 15-point Likert scale, ranging from −7 (a very great deal worse) to +7 (a very great deal better).13
Study procedures
A study-specific online module was designed within the Zorgdraad integrated care IT system. The CAT and CCQ were designed to appear similar to the paper versions. The order in which the CAT or CCQ was presented to the patient was randomized to eliminate any completing fatigue effects. The time between loading of the webpage and submitting the form was recorded electronically. The easiness to complete, understand the questions, questionnaire preference, and importance of issues addressed were assessed using additional questions.
Within 1 week after completing the electronic version, the participant completed a paper copy of the CAT and the CCQ at their homes. In addition, the GRC was completed to assess stability of the disease. For comparison of the electronic and paper versions, only stable patients with a GRC score between −1 to +1 were included in the analysis.
Sample size and statistical analysis
A sample size calculation using data from Ringbaek and colleagues8 with a power of 80% and a two-sided significance level of 5% indicated that 88 patients were needed to complete the study to show at least a 30 s difference in filling out time, which difference we considered clinically relevant.
Difference in time to complete the electronic version of the CAT and the electronic version of the CCQ was tested using a paired sample t-test. In the event assumptions of the t-test were violated, testing was conducted on logtransformed variables or, if unsuccessful, Mann–Withney U tests. Data were descriptively presented as medians with IQRs.
The differences between the electronic versions of the CAT and the CCQ regarding easiness to complete, importance of issues raised in questionnaires, and the importance of information for health-care provider were determined using paired sample t-tests, or, when assumptions were violated, paired sample t-tests on logtransformed variables. If logtransformation was unsuccessful Wilcoxon signed-rank tests were used. The intraclass correlation coefficient (ICC) and Bland–Altman plots were used to assess agreement between the electronic version and the paper version of the CAT and CCQ.
The study was registered at the Dutch Trial Register (NTR3384), and The University Medical Center Groningen Ethics Board approved the study.
References
GOLD Committee. Global strategy for diagnosis, management, and prevention of COPD. http://goldcopd.org/archived-reports/. Updated February 2013.
van der Molen, T., Miravitlles, M. & Kocks, J. W. COPD management: role of symptom assessment in routine clinical practice. Int. J. Chron. Obstruct. Pulmon. Dis. 8, 461–471, doi:10.2147/COPD.S49392 (2013).
Jones, P. W., Harding, G., Berry, P., Wiklund, I., Chen, W. H. & Kline, L. N. Development and first validation of the COPD assessment test. Eur. Respir. J. 34, 648–654 (2009).
Kocks, J., de Jong, C., Berger, M. Y., Kerstjens, H. A., van der Molen, T. Putting health status guided COPD management to the test: protocol of the MARCH study. BMC Pulm. Med. doi:10.1186/1471-2466-13-41 (2013).
GOLD Committee. Global strategy for the diagnosis, management and prevention of COPD. Global initiative for chronic obstructive lung disease (GOLD). http://goldcopd.org/archived-reports/. Update 2011.
Bestall, J. C., Paul, E. A., Garrod, R., Garnham, R., Jones, P. W. & Wedzicha, J. A. Usefulness of the medical research council (MRC) dyspnoea scale as a measure of disability in patients with chronic obstructive pulmonary disease. Thorax 54, 581–586 (1999).
van der Molen, T., Willemse, B. W., Schokker, S., ten Hacken, N. H., Postma, D. S. & Juniper, E. F. Development, validity and responsiveness of the clinical COPD questionnaire. Health Qual. Life Outcomes 1, 13 (2003).
Ringbaek, T., Martinez, G. & Lange, P. A comparison of the assessment of quality of life with CAT, CCQ, and SGRQ in COPD patients participating in pulmonary rehabilitation. Chron. Obstruct. Pulmon. Dis. 9, 12–15, doi:10.3109/15412555.2011.630248 (2012).
Sundh, J., Stallberg, B., Lisspers, K., Kampe, M., Janson, C. & Montgomery, S. Comparison of the COPD assessment test (CAT) and the clinical COPD questionnaire (CCQ) in a clinical population. Chron. Obstruct. Pulmon. Dis. 13, 57–65, doi:10.3109/15412555.2015.1043426 (2016).
Tsiligianni, I. G. et al. Assessing health status in COPD. A head-to-head comparison between the COPD assessment test (CAT) and the clinical COPD questionnaire (CCQ). BMC Pulm. Med. 12, 20, doi:10.1186/1471-2466-12-20 (2012).
Cave, A. J., Atkinson, L., Tsiligianni, I. G. & Kaplan, A. G. Assessment of COPD wellness tools for use in primary care: an IPCRG initiative. Int. J. Chron. Obstruct. Pulmon. Dis. 7, 447–456, doi:10.2147/COPD.S29868 (2012).
Smeele, I. et al. M26. NHG-standaard COPD. Huisarts Wet. 50, 362–379 (2007).
Juniper, E. F., Guyatt, G. H., Willan, A. & Griffith, L. E. Determining a minimal important change in a disease-specific quality of life questionnaire. J. Clin. Epidemiol. 47, 81–87 (1994).
Acknowledgements
Funding
This study was financially supported by an unrestricted research grant by GSK NL. Publication was financially supported by the International Primary Care Respiratory Group and the Primary Care Respiratory Society UK.
Author information
Authors and Affiliations
Contributions
K.J.W.H. was involved in the design, data collection, analysis, interpretation, and wrote the first draft. B.C.M.G. was involved in the design and data collection. K.M.J. was involved in analysis and interpretation. O.W. was involved in the design and data collection. K.B.J. was involved in the design, data collection, and analysis. V.d.M.T. was involved in the design, data collection, and interpretation. C.N.H. was involved in the design, data collection, analysis, and interpretation. All authors contributed to the writing of the paper. K.J.W.H. and C.N.H. are the guarantors of the paper.
Corresponding author
Ethics declarations
Competing interests
V.d.M.T. developed the CCQ and holds the CCQ’s copyright. The remaining authors declare no competing interests.
Rights and permissions
This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
About this article
Cite this article
Kocks, J., Blom, C., Kasteleyn, M. et al. Feasibility and applicability of the paper and electronic COPD assessment test (CAT) and the clinical COPD questionnaire (CCQ) in primary care: a clinimetric study. npj Prim Care Resp Med 27, 20 (2017). https://doi.org/10.1038/s41533-017-0023-0
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41533-017-0023-0
This article is cited by
-
Role of the active cycle of breathing technique combined with phonophoresis for the treatment of patients with chronic obstructive pulmonary disease (COPD): study protocol for a preliminary randomized controlled trial
Trials (2021)
-
Assessing health status over time: impact of recall period and anchor question on the minimal clinically important difference of copd health status tools
Health and Quality of Life Outcomes (2018)