Abstract
Background:
The gold standard for the diagnosis of chronic obstructive pulmonary disease (COPD) is spirometry, but there are barriers to its use in primary care.
Aims:
To externally validate the COPD Diagnostic Questionnaire (CDQ) as a diagnostic tool in patients at increased risk in Australian general practice and to compare its performance with other CDQ validation studies.
Methods:
Patients were recruited from 36 general practices in Sydney, Australia. Former or current smokers aged 40–85 years with no prior COPD diagnosis were invited to a case-finding appointment with the practice nurse. The CDQ was collected and pre- and postbronchodilator spirometry was performed. Cases for whom complete CDQ data were present and the spirometry met quality standards were analysed.
Results:
Of 1,631 patients who attended case-finding recruitment, 1,054 (65%) could be analysed. Spirometry showed 13% had COPD. The ability of the CDQ to discriminate between patients with and without COPD was fair, represented by the area under the receiver operating characteristic curve of 0.713. With a CDQ cut-off point value of 16.5 the sensitivity was 80% and specificity 47% and, at a cut-off point value of 19.5, the sensitivity was 63% and specificity 70%.
Conclusions:
The CDQ did not discriminate between patients with and without COPD accurately enough to use as a diagnostic tool in patients at increased risk of COPD in Australian general practice. Further research is needed on the value of the CDQ as a tool for selecting patients for spirometry.
Similar content being viewed by others
Introduction
Chronic obstructive pulmonary disease (COPD) is a common respiratory disease characterised by airflow obstruction. The major risk factor in developed countries is tobacco smoking.1 COPD is an important cause of mortality and disability worldwide, ranked globally as the fifth leading cause of death and 11th leading cause of disability-adjusted life years lost in 2002.2 In Australia in 2007, COPD was the fourth and sixth leading causes of death for men and women, respectively, and made up 1% of all hospitalisations.3 In a recent multi-regional Australian study, 14.5% of the population aged ≥40 years had COPD based on spirometric diagnosis, with the prevalence higher in older age groups.4
The gold standard for COPD diagnosis is post-bronchodilator (post-BD) spirometry assessment in the clinical context of any patient with dyspnoea, chronic cough, or sputum production with a history of exposure to risk factors such as tobacco smoke and occupational dusts.5 Performing screening spirometry in the general population — which includes asymptomatic people — is controversial, with insufficient evidence to indicate that screening improves morbidity, mortality, or smoking cessation rates.5,6 The barriers to spirometry use in general practice include lack of expertise in performing spirometry, poor access to a well maintained spirometer, the time consuming nature of pre- and post-BD spirometry, and low confidence in spirometry interpretation.7 This can lead to underdiagnosis and misdiagnosis of COPD, particularly if general practitioners rely on a symptom-based assessment.7,8 High-quality management of COPD requires accurate diagnosis to relieve symptoms, improve health status, prevent exacerbations and disease progression, and reduce early mortality.5
Although the CDQ was not developed as a diagnostic tool, it has been validated in comparison with spirometry in subjects selected from primary care settings in Europe and Australia and hospital outpatient clinics in Japan.15,19–21 Two of these studies investigated the validity of the Piko-6® flow meter (a form of micro-spirometry) as a screening tool for COPD diagnosis in primary care.15,21 These studies also validated the CDQ and compared it with micro-spirometry.15,21 Other questionnaires such as the COPD Population Screener Questionnaire (COPD-PS) and a condensed version of the COPD Assessment Test (CAT) have been developed but have not undergone external validation.13,14
The aim of this study was to validate the CDQ externally in a large sample of current and former smokers with no prior diagnosis of obstructive lung disease recruited from general practices in Sydney, Australia. We wanted to determine if the CDQ could be used as a COPD diagnostic tool in patients at increased risk in Australian general practice. This complements an earlier CDQ study in a smaller cohort recruited from Australian general practices.21 The analysis methods were based on a protocol outlined by Price et al.12 The study measured the relationship between CDQ scores and COPD proportions determined by spirometry and these results were compared with the findings in the study by Price et al., one Australian and three international external validation studies.12,15,19–21 The variability in how the CDQ performed in this and other external validation studies was examined.
Methods
Patient recruitment for external validation
Patients in this validation study were from a case-finding recruitment group for a cluster randomised controlled trial of early intervention in COPD by practice nurse-general practitioner teams. Written informed consent was obtained from all patients. The details of the protocol of this trial have been described elsewhere.22 Ethics approval was granted by the University of New South Wales Human Research Ethics Committee.
Patients aged 40–85 years who were former or current smokers with no previous diagnosis of COPD or other obstructive lung disease were invited to a case-finding appointment with a practice nurse in one of the 36 study general practices. The nurses had attended eight hours of training in spirometry and how to administer the CDQ.22 The CDQ was completed by the practice nurse prior to performing pre- and post-BD spirometry using 400μg salbutamol or 500μg terbutaline (for those refusing salbutamol) via a metered dose inhaler based on the American Thoracic Society and the European Respiratory Society (ATS/ERS) 2005 lung function guidelines.23,24 The practice nurses used the practice's own spirometer which had been calibrated by the research team. Several different models of spirometer were used across the study, with each model being found in more than one practice. Spirometry tracings were independently reviewed by a respiratory physiologist (AJC). Cases where spirometry met quality standards based on the ATS/ERS 2005 criteria and for whom complete CDQ data were present were included in the analysis.24 A study diagnosis of COPD was assigned to subjects who had a post-BD forced expiratory volume in one second/forced vital capacity (FEV1/FVC) ratio <0.7, which was based on the Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines.5 Calculation of the CDQ scores was performed by the project officer using predetermined scoring criteria.12
CDQ score and statistical analysis
A three-tier scoring system was used to assign subjects into groups of low, intermediate, and high likelihood of COPD based on the CDQ score 0–38 with cut-off points at 16.5 and 19.5.12 The receiver operating characteristic area under the curve (ROCAUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for the two cut-off points using the non-parametric method. The Pearson χ2 test and independent sample t tests were used for comparing demographic information between included and excluded subject groups. A p value of <0.05 indicated a statistically significant difference. The raw CDQ score was used as the screening test variable and the COPD diagnosis as the dichotomised classification variable. Analyses were performed using SPSS (IBM, Armonk, NY, USA) and Microsoft Excel (Microsoft, Redmond, WA, USA) software.
Results
Study population selection
A total of 1,631 people attended for case-finding recruitment. Of these, 1,054 (65%) had the complete CDQ recorded and spirometry meeting quality criteria for analysis. Figure 1 shows the reasons for exclusion of the remaining 35% of subjects. The mean±SD age (62.5±11.6 years vs. 61.0±11.3 years, p=0.009) and percentage of males (58.9% vs. 51.8%, p=0.006) was significantly higher for excluded patients than for the included group. There was no significant difference between excluded and included groups in current smoking status (22.7% vs. 22.3%, p=0.825).
Characteristics of participants
Complete data for 1,054 patients were used in the study group analyses. Table 1 shows the population characteristics of the two groups based on COPD diagnosis. After post-BD spirometry, 13.1% of the total population were diagnosed with COPD. The mean post-BD FEV1/FVC ratio for the COPD group was 63% compared with 80% for the non-COPD group and 78% for all subjects. The COPD group had more men, a higher proportion of current smokers and CDQ scores (by four points), and were on average five years older than the non-COPD group.
Performance of the CDQ in this study and comparison with other CDQ validation studies
Following application of the two cut-off points of 16.5 and 19.5 after CDQ calculation in accordance with the original CDQ study,12 43.4% had a low likelihood of COPD (<16.5), 22.4% had an intermediate likelihood (16.5–19.5), and 34.3% of patients had a high likelihood of COPD (>19.5). In terms of the distribution of subjects with COPD within the CDQ zones, 20% had scores in the lowest zone but 63% had scores in the highest zone (Table 1). Within the three zones, 6.1% of subjects had COPD in the low likelihood zone (28 of 457 subjects), 9.7% in the middle zone (23 of 236), and 24.1% (87 of 361) in the high zone.
The trend of increasing COPD proportions from the low to the high zone was published in three of the studies, as shown in Table 2. The ability of the CDQ to discriminate between patients with and without COPD is represented by the ROC curve (Figure 2). The ROCAUC was 0.713, which is lower than the original study and two other validation studies.12,19,21 This is shown in Tables 2 and 3, with results of the other studies derived from their respective papers.12,15,19–21
Sensitivity and specificity at the cut-off point value of 16.5 (cut-off point A) were 79.7% and 46.8%, respectively and, at 19.5 (cut-off point B), the sensitivity and specificity were 63.0% and 70.1%, respectively. When compared with other validation studies (Table 3), sensitivity was lower at both cut-off points. However, with the exception of the study by Sichletidis et al., the specificity was higher.15 The PPV at cut-off points A and B were 18.4% and 24.1% for this study compared with 30.3% and 37%, respectively, in the original study.12 The NPV was comparable to the original study at both cut-off points (93.9% this study vs. 92.7% in the original study at point A and 92.6% vs. 89.0% at B).12
Discussion
Main findings
In this study, using the three-tier scoring system, the CDQ did not perform well in identifying people with COPD when compared with spirometry. The ROCAUC of 0.713 is fair, and is higher than that in the study by Kotz et al. who considered their ROCAUC of 0.65 to be very low.20,25 The ROCAUC in this study is less than two other external validation studies and less than the original study which had an ROCAUC of 0.816.12,19,21 The ROCAUC in this study is closer to 0.5 than 1.0, where a test with an area under the curve of 1 would represent a perfect test with no overlap between true positives and false positives and the optimal operating point corresponding to the upper left-hand corner of the ROC graph (Figure 2).26 An ROCAUC of 0.5 indicates a test with no discriminative power and would be essentially worthless.25,26
The ability of the CDQ to identify patients with and without COPD varies between populations. This is influenced by characteristics such as smoking and age. When applying the two cut-off points from the CDQ in this study, the questionnaire achieved sensitivities of 79.7% and 63% at cut-off points A (16.5) and B (19.5), with specificities of 46.8% and 70.1%, respectively. This means that, in this population, at cut-off point A about 80% of patients with a COPD diagnosis on spirometry were correctly identified by the questionnaire, but 55% of patients without COPD were incorrectly identified by the CDQ as having COPD. At cut-off point B, while about two thirds of patients with COPD were correctly identified by the CDQ, 30% of patients without COPD were incorrectly identified as having COPD. The PPV of 24.1% at cut-off point B indicates that a person with a CDQ score above 19 has a one in four chance of spirometry-diagnosed COPD. When designing the CDQ, Price et al. considered that a reasonable PPV would be at least 50% for the higher cut-off point, but the PPV at cut-off point B was less than 50% in the three studies which published these results (Table 3).12
Strengths and limitation of this study
A particular strength of the study is that it is a large sample of primary care patients from Australia made up of former and current smokers, the number of participants only exceeded by the study by Sichletidis et al.15 The number of patients excluded from the study may have biased the outcome. For example, the invalid subset was older by an average of 1.5 years. Potentially this could have increased the proportion of COPD in the final population as higher CDQ scores reflect higher COPD prevalence and COPD diagnoses increase with age.4 The proportion of subjects excluded from the statistical analysis (i.e. 35%) is almost double the cases excluded from the CDQ validation study by Kotz et al.20 (18%) and substantially higher than the Greek and Australian external validity studies as well as the original study (9%).12,19,21 This could be due to the different operators at each of the 36 sites for spirometry and the use of several different spirometry models, although every practice nurse received spirometric training. The other studies used independently trained operators such as respiratory physicians or research assistants to perform the spirometry and each study used one spirometer model/brand. While the latter design may help to reduce variability in spirometry performance and improve interpretation of adequate spirometry attempts as per the ATS/ERS 2005 criteria, this study's design better represents real-world use of the CDQ and spirometry in Australian general practice.
As our sample included patients without respiratory symptoms, this may underestimate the ability of the CDQ to detect cases of COPD as defined by the GOLD guidelines.5 This limitation also applies to the other CDQ studies that looked at detecting spirometrically-defined COPD in patients with risk factors for COPD but not necessarily symptoms.12,15,19–21 Given the controversy of detecting asymptomatic airflow obstruction, this could be seen as an advantage of the CDQ in that asymptomatic patients would be less likely to score above the CDQ cut-off points. Further research is needed to determine if the CDQ performs better as a diagnostic tool for detecting only those cases meeting all the GOLD criteria.
Interpretation of findings in relation to previously published work
Two of the validation studies compared micro-spirometry with full spirometry, with or without the CDQ. Sichletidis et al. used post-BD micro-spirometry with the Piko-6® meter whereas Frith et al. used this device in the pre-BD stage prior to full spirometry.15,21 The Piko-6® meter proved to be a more specific and less sensitive test, with a higher PPV than CDQ for COPD diagnosis. The ROCAUC of 0.85 for the Piko-6® meter in the study by Frith et al. outperformed the CDQ across all studies.21 The performance of the CDQ/Piko-6® combination vs. spirometry in the study by Sichletidis et al. reduced the sensitivity and increased the specificity compared with the tests individually.15 Frith et al. concluded that measuring FEV1/FEV6 from a simple flow meter could optimise early referral for spirometry.21 Using a pre-BD peak flow meter to screen individuals with respiratory symptoms and at high risk of COPD for diagnostic spirometry has been discussed previously but has not been externally validated.27
Another possible role for the CDQ is as a selection tool for spirometry, removing the need for this test in patients at low risk of COPD. The concept of using the CDQ to ‘pre-screen’ those smokers at highest likelihood of COPD for spirometry using the higher cut-off point instead of performing spirometry on all current and former smokers was discussed previously by Price et al.12 Subjects in the intermediate zone of likelihood in the CDQ could undergo spirometry but, where there were limited resources to do spirometries in this group, these subjects could be followed up clinically and spirometry deferred to a later date to minimise the number of unnecessary spirometries.12 The International Primary Care Respiratory Group (IPCRG) have recommended a diagnostic process where all patients over 35 years of age should be evaluated for their risk of developing COPD by completing the CDQ and/or ‘case-identification’ spirometry prior to standard diagnostic spirometry.28 The IPCRG assigned the lower cut-off point as a singular cut-off point for the CDQ in this diagnostic process, with subjects scoring ≥17 going on to have diagnostic spirometry.12,28
Implications for future research, policy and practice
Further research is needed on the use of the CDQ as a selection tool for proceeding to spirometry. This could potentially save both time and money if found to be effective. It may be more practical to set one instead of two cut-off points for this application of the questionnaire. At the lower cut-off point the sensitivity of 80% would be considered suboptimal when compared with the other external validation CDQ studies with sensitivities ranging between 89% and 94% (Table 3). If high sensitivity is preferable for diagnosing more COPD cases, then the cut-off point can be set as low as is practical, but at the expense of more negative spirometry tests. Alternatively, one could select a cut-off point that finds the right balance between sensitivity and specificity for selecting patients for spirometry in a given setting. Therefore, in a future study it would be important to look at how the CDQ performs using a two-tier versus a three-tier scoring system.
Conclusions
The results of this external validation study suggest that the questionnaire does not discriminate between patients with and without spirometrically-defined COPD accurately enough to use as a stand-alone diagnostic tool in Australian general practice. Further research is needed on the value of the CDQ as a tool for selecting patients to proceed to spirometry. Setting one instead of two cut-off points for the CDQ could make it more practically applicable for this purpose.
References
Mannino DM, Buist AS . Global burden of COPD: risk factors, prevalence, and future trends. Lancet 2007;370:765–73. http://dx.doi.org/10.1016/S0140-6736(07)61380-4
Mathers CD, Loncar D . Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med 2006;3(11):e442;2011–30. http://dx.doi.org/10.1371/journal.pmed.0030442
Australian Institute of Health and Welfare. Australia's health 2010. Australia's health series no. 12. Cat. no. AUS 122. Canberra: Australian Institute of Health and Welfare, 2010.
Toelle BG, Xuan W, Bird TE, et al. Respiratory symptoms and illness in older Australians: the Burden of Obstructive Lung Disease (BOLD) study. MJA 2013;198(3):144–8. http://dx.doi.org/10.5694/mja11.11640
Global Initiative for Chronic Obstructive Lung Disease. Global strategy for the diagnosis, management and prevention of chronic obstructive pulmonary disease. http://www.goldcopd.org/uploads/users/files/GOLD_Report_2013_Feb20.pdf. Updated 2013 (accessed 13 August 2013).
Lin K, Watkins B, Johnson T, Rodriguez JA, Barton MB . Screening for chronic obstructive pulmonary disease using spirometry: summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med 2008;148(7):535–43. http:dx.doi.org/10.7326/0003-4819-148-7-200804010-00213
Walters JAE, Hansen E, Mudge P, Johns DP, Haydn Walters E, Wood-Baker R . Barriers to the use of spirometry in general practice. Aust Fam Physician 2005;34(3):1–3.
Walters JA, Haydn Walters E, Nelson M, et al. Factors associated with misdiagnosis of COPD in primary care. Prim Care Respir J 2011;20(4):396–402. http://dx.doi.org/10.4104/pcrj.2011.00039
Freeman D, Nordyke R, Isonaka S, et al. Questions for COPD diagnostic screening in a primary care setting. Respir Med 2005;99:1311–18. http://dx.doi.org/10.1016/j.rmed.2005.02.037
Kida K, Wakabayashi R, Mizuuchi T, Murata A . Screening for suspected chronic obstructive pulmonary disease with an eleven-item pre-interview questionnaire (11-Q). Intern Med 2006;45:1201–07. http://dx.doi.org/10.2169/internalmedicine.45.1694
Price DB, Tinkelman DG, Halbert RJ, et al. Symptom-based questionnaire for identifying COPD in smokers. Respiration 2006;73:285–95. http://dx.doi.org/10.1159/000090142
Price DB, Tinkelman DG, Nordyke RJ, Isonaka S, Halbert RJ; for the COPD Questionnaire Study Group. Scoring system and clinical application of COPD diagnostic questionnaires. Chest 2006;129(6):1531–9. http://dx.doi.org/10.1378/chest.129.6.1531
Martinez FJ, Raczek AE, Selfer FD, et al. Development and initial validation of a self-scored COPD Population Screener questionnaire (COPD-PS). COPD 2008;5:85–95. http://dx.doi.org/10.1080/15412550801940721
Raghavan N, Lam Y-M, Webb KA, et al. Components of the COPD Assessment Test (CAT) associated with a diagnosis of COPD in a random population sample. COPD 2012;9:175–83. http://dx.doi.org/10.3109/15412555.2011.650802
Sichletidis L, Spyratos D, Papaioannou M, et al. A combination of the IPAG questionnaire and PiKo-6® flow meter is a valuable screening tool for COPD in the primary care setting. Prim Care Respir J 2011;20(2):184–9. http://dx.doi.org/10.4104/pcrj.2011.00038
Dirven JAM, Muris JWM, van Schayck CP. COPD screening in general practice using a telephone questionnaire. COPD 2010;7:352–9. http://dx.doi.org/10.3109/15412555.2010.510547
Dirven JAM, Tange HJ, Muris JWM, van Haaren KMA, Vink G, van Schayck OCP. Early detection of COPD in general practice: patient or practice managed? A randomised controlled trial of two strategies in different socioeconomic environments. Prim Care Respir J 2013;22(3):331–7. http://dx.doi.org/10.4104/pcrj.2013.00070
Dirven JAM, Tange HJ, Muris JWM, van Haaren KMA, Vink G, van Schayck OCP. Early detection of COPD in general practice: implementation, workload and socioeconomic status. A mixed methods observational study. Prim Care Respir J 2013;22(3):338–43. http://dx.doi.org/10.4104/pcrj.2013.00071
Kawayama T, Minakata Y, Matsunaga K, et al. Validation of symptom-based COPD questionnaires in Japanese subjects. Respirology 2008;13:420–6. http://dx.doi.org/10.1111/j.1440-1843.2008.01241.x
Kotz D, Nelemans P, van Schayck CP, Wesseling GJ . External validation of a COPD diagnostic questionnaire. Eur Respir J 2008;31(2):298–303. http://dx.doi.org/10.1183/09031936.00074307
Frith P, Crockett A, Beilby J, et al. Simplified COPD screening: validation of the PiKo-6® in primary care. Prim Care Respir J 2011;20(2):190–8. http://dx.doi.org/10.4104/pcrj.2011.00040
Bunker JM, Reddel HK, Dennis SM, et al. A pragmatic cluster randomized controlled trial of early intervention for chronic obstructive pulmonary disease by practice nurse-general practitioner teams: study protocol. Implement Sci 2012;7:83. http://dx.doi.org/10.1186/1748-5908-7-83
Pellegrino R, Viegi G, Brusasco V, et al. Interpretative strategies for lung function tests. Eur Respir J 2005;26(5):948–68. http://dx.doi.org/10.1183/09031936.05.00035205
Miller MR, Hankinson J, Brusasco V, et al. Standardisation of spirometry. Eur Respir J 2005;26(2):319–38. http://dx.doi.org/10.1183/09031936.05.00034805
Tape TG . Interpreting diagnostic tests. http://gim.unmc.edu/dxtests/Default.htm (accessed 13 August 2013).
van Erkel AR, Pattynama PM . Receiver operating characteristic (ROC) analysis: basic principles and applications in radiology. Eur J Radiol 1998;27:88–94. http://dx.doi.org/10.1016/S0720-048X(97)00157-5
Jithoo A, Enright PL, Burney P, et al. Case-finding options for COPD: results from the Burden of Obstructive Lung Disease Study. Eur Respir J 2013;41(3):548–55. http://dx.doi.org/10.1183/09031936.00132011
Price D, Crockett A, Arne M, et al. Spirometry in primary care case-identification, diagnosis and management of COPD. Prim Care Respir J 2009;18(3):216–23. http://dx.doi.org/10.4104/pcrj.2009.00055.
Acknowledgements
Handling editor Anthony D'Urzo
Statistical review Gopal Netuveli
This paper honours the memory of Dr Jeremy Bunker, without whose initiative and enthusiasm the trial from which the results are derived would not have taken place. We thank the practice nurses who participated in the study.
Funding This study was funded by the National Health and Medical Research Council, Project Grant No. 630421.
Author information
Authors and Affiliations
Contributions
The original trial on which this study was based was conceived by NAZ, Dr Jeremy Bunker and Professor Guy Marks and all authors contributed to this study's design, either through the original trial or the current study. OCPvS advised on the design of the CDQ. AJC designed the spirometry toolkit for diagnosis of COPD and performed quality assessment of the spirometry. AJS was the primary author and performed the bulk of the statistical analysis, aided by IH. All authors contributed to and approved the final version of the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare they have no conflicts of interest in relation to this article. OCPvS is an Assistant Editor and AJC is on the International Editorial Board of PCRJ, but they were not involved in the editorial review of or the decision to publish this article.
Appendix 1. COPD Diagnostic Questionnaire (CDQ)
Appendix 1. COPD Diagnostic Questionnaire (CDQ)
Rights and permissions
About this article
Cite this article
Stanley, A., Hasan, I., Crockett, A. et al. Validation of the COPD Diagnostic Questionnaire in an Australian general practice cohort: a cross-sectional study. Prim Care Respir J 23, 92–97 (2014). https://doi.org/10.4104/pcrj.2014.00015
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.4104/pcrj.2014.00015
This article is cited by
-
Specialist respiratory outreach: a case-finding initiative for identifying undiagnosed COPD in primary care
npj Primary Care Respiratory Medicine (2021)
-
Case-finding for COPD clinic acceptability to patients in GPs across Hampshire: a qualitative study
npj Primary Care Respiratory Medicine (2021)
-
An accurate prediction model to identify undiagnosed at-risk patients with COPD: a cross-sectional case-finding study
npj Primary Care Respiratory Medicine (2019)
-
Development and validation of the Salzburg COPD-screening questionnaire (SCSQ): a questionnaire development and validation study
npj Primary Care Respiratory Medicine (2017)
-
The RESPECT study: RESearch on the PrEvalence and the diagnosis of COPD and its Tobacco-related etiology: a study protocol
BMC Public Health (2015)