Substantial over-diagnosis and under-diagnosis of asthma in adults and children has recently been reported. As asthma is mostly diagnosed in non-specialist settings, a clinical prediction model (CPM) to aid the diagnosis of asthma in primary care may help improve diagnostic accuracy. We aim to systematically identify, describe, compare, and synthesise existing CPMs designed to support the diagnosis of asthma in children and adults presenting with symptoms suggestive of the disease, in primary care settings or equivalent populations. We will systematically search Medline, Embase and CINAHL from 1 January 1990 to present. Any CPM derived for use in a primary care population will be included. Equivalent populations in countries without a developed primary care service will also be included. The probability of asthma diagnosis will be the primary outcome. We will include CPMs designed for use in clinical practice to aid the diagnostic decision making of a healthcare professional during the assessment of an individual with symptoms suggestive of asthma. We will include derivation studies, and external model validation studies. Two reviewers will independently screen titles/abstracts and full texts for eligibility and extract data from included papers. The CHARMS checklist (or PROBAST if available) will be used to assess risk of bias within each study. Results will be summarised by narrative synthesis with meta-analyses completed if possible. This systematic review will provide comprehensive information about existing CPMs for the diagnosis of asthma in primary care and will inform the development of a future diagnostic model.
Asthma is a chronic respiratory disease affecting an estimated 334 million people worldwide.1 Progress towards improving asthma outcomes has been limited over the past decade.2 A key determinant in providing optimal asthma care is accurate and timely diagnosis. Yet, asthma remains a challenging disease to diagnose and substantial over-diagnosis and under-diagnosis of asthma in adults and children is reported.3,4,5 The consequences of under-diagnosis are lack of treatment, avoidable morbidity and mortality; while over-diagnosis leads to costly, potentially harmful treatment and unnecessary healthcare use. Furthermore, to realise the potential of precision medicine, greater accuracy in determining asthma traits at the point of diagnosis is required.2
Accurate diagnosis is challenging in a condition defined by its variability, in which the diagnosis is clinical, and in which symptoms, signs, and tests have poor predictive value for the diagnosis.6,7,8 The heterogeneity of asthma and recognition of distinct phenotypes and endotypes, adds further complexity,2,9 particularly as variable airflow obstruction remains the core diagnostic feature of asthma in current guidelines.7,8
Clinical prediction models (CPM), also referred to as prediction rules or risk scores10 combine at least two predictors, such as elements from a clinical history, physical examination, test results or response to treatment, to estimate the probability that a certain outcome is present.10,11 In clinical practice, CPMs can assist healthcare professionals to weigh up the probability of a diagnosis, enhance decision making with patients, or aid patient stratification.12,13,14
The current diagnostic strategies recommended in asthma guidelines are based on expert consensus and in some instances informed by health economic modelling, yet the accuracy with which the advocated clinical assessment pathways result in a correct diagnosis have not been formally evaluated.7,8,15
A prospectively validated CPM to support the diagnosis of asthma would offer an evidence-based approach for the assessment of individuals with symptoms suggestive of asthma. As most asthma diagnoses occur in non-specialist settings,2 a CPM developed for use in primary care (or equivalent settings where undifferentiated health problems are presented to healthcare professionals)16 would be particularly valuable.
To our knowledge, there are few CPMs to support the diagnosis of asthma in primary care. Previous searches identified a symptom-based diagnostic tool from South Korea,17 and an algorithm derived from Japanese healthcare;18 neither have been externally validated. A decision tree used to differentiate obstructive respiratory conditions has been derived and validated in the Netherlands.19 To build on these foundations, inform future development and progress towards wider implementation of a CPM for asthma diagnosis in primary care, a systematic review is needed.
To identify, describe, compare, and synthesise all CPMs designed to support the diagnosis of asthma in children and adults presenting with symptoms suggestive of the disease, in primary care settings or equivalent populations.
Accurately diagnosing asthma is pivotal for effective symptom control and prevention of asthma attacks. By conducting this systematic review of CPMs for the diagnosis of asthma in primary care, we will critically appraise each included model, understand the strengths and limitations of each study design and the utility of each candidate predictor used, including signs and symptoms, personal and family history, and results from past and current clinical tests.
Although a CPM offers potential for improving the diagnosis of asthma, a number of questions need to be addressed before it will be accepted into clinical practice. Of fundamental importance is the outcome measure used to derive the model. In the absence of a “golden” reference standard for asthma diagnosis, the outcome measure must be carefully considered if an accurate diagnosis is to be made. Second, regardless of the statistical accuracy of a model, to be clinically useful, its impact on clinical outcomes should be tested, and healthcare professionals must trust the model and want to use it.20,21 Finally, contemporary understanding of asthma as an aggregate diagnosis2,22 suggests that a future diagnostic CPM may align with a precision medicine approach, potentially being able to stratify individuals by treatable traits.9 Our systematic approach will provide comprehensive information about existing CPMs, and inform the future development of a model for the diagnosis of asthma in primary care.
The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) has been used to define the review question, and develop the review design.23 The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guided protocol reporting.24
Data sources and search strategy
We will complete a systematic search to identify all studies describing CPMs to support the diagnosis of asthma in primary care settings. In this review, a CPM is defined as any tool that combines at least two predictors, to provide an estimate for the probability of an outcome.10,14
Medline, Embase, and CINAHL databases will be searched from 1 January 1990 to present. The search strategy (Appendix 1) uses Cochrane asthma search syntax, with established search filters for prediction models.25,26 CPMs published in national clinical guidelines will be sought in TRIP (https://www.tripdatabase.com) and Guidelines Clearinghouse (https://www.guideline.gov). Forward and backward citations from the included studies will be searched for additional eligible studies. International experts and authors known to have published relevant work will be contacted to identify further studies. No language restrictions will be used. All studies will be translated into English where possible, any literature that we are unable to translate will be declared.
Study eligibility criteria
Types of study
We will include derivation studies with or without external validation, in addition to external model validation studies with or without model updating.23
Any CPM derived for use in a primary care population (or equivalent) will be included.
Intervention (type of model)
We will include CPMs designed for use in clinical practice to aid the diagnostic decision making of a healthcare professional during the assessment of an individual with symptoms suggestive of asthma.
There are no relevant comparators for this systematic review.
Outcome to be predicted
The primary outcome is the probability of asthma diagnosis. Included studies should present at least one prediction model, or equivalent statistical method, in such a way that allows the probability of asthma to be calculated for other individuals. Some CPMs may use the broader outcome of obstructive airways disease. These studies will be included if data pertaining to asthma diagnosis can be extracted. CPMs derived for the diagnosis of occupational asthma will be included if the intended use is to aid diagnostic decision making in patients with undifferentiated symptoms.
Reference standard used to diagnose asthma
From scoping work, we anticipate that studies designed to derive and validate asthma diagnosis CPMs will use a variety of reference standards. We will evaluate the impact of using different measures as the reference standard. However, studies which do not use an internationally accepted definition for asthma will be excluded.
Studies will be excluded if:
The predictive value of more than one variable was evaluated but not combined to produce a diagnostic estimate.
Publication occurred before 1 January 1990.
Variables used in the model are not available in routine clinical practice or not clearly reported.
Outcomes for asthma are not separate or data relating to the asthma outcome is not extractable.
The CPM was derived to predict future risk of asthma.
Over half of the participants included were children less than 5 years old (because of the overlap between asthma and viral-associated wheeze in this age group).
The reference standard used is not based on an internationally recognised definition of asthma.
Non-original studies such as editorials, expert views, non-research letters.
Data management and selection process
Records will be retrieved from databases and transferred into a reference manager. Records will be de-duplicated, screened, and managed using Covidence (https://www.covidence.org). Two reviewers (L.D., A.B.) will independently screen titles and abstracts for eligibility. Full-text copies of all potentially relevant records will be obtained. Two reviewers (L.D., A.B. or S.McL.) will independently assess whether each full-text record meets the inclusion criteria. Any discrepancies will be arbitrated by a third reviewer (H.P., S.L., or A.S.).
Included articles will be categorised into two groups: derivation studies and external model validation (with or without model updating). A standardised data extraction form to collect relevant data items from eligible studies will be developed and piloted in line with Cochrane and CHARMS guidance.23 Data extraction will be undertaken by two reviewers (L.D., S.McL.) who will independently extract relevant data from each included study onto the data extraction form. Any disagreements will be resolved by a third reviewer (H.P., S.L., or A.S.).
Data from the included studies will be summarised in descriptive tables. Table 1 lists the minimum set of data items to be extracted. If a study reports more than one model, we will extract data separately for each model.
Critical appraisal and risk of bias in individual studies
The CHARMS checklist will guide risk of bias assessment within individual studies.23 The prediction model risk of bias assessment tool (PROBAST) will be used if available.27 Quality assessment will be used to evaluate whether study quality makes any impression on the conclusions.
Results will be summarised by narrative synthesis and descriptive statistics. Methods for meta-analysis of model validation studies have recently been reported28 and will be considered if suitable studies are identified.
Evaluating confidence in cumulative evidence
We will evaluate the overall evidence and strength of recommendations gathered from the review using the Grading of Recommendations Assessment, Development and Evaluation system.31
A protocol will be registered with the International Prospective Register of Systematic Reviews (PROSPERO). In the event of a protocol amendment, the date, description, and rationale will be reported.
Data sharing is not applicable as this article describes a protocol and no datasets were generated or analysed.
Global Asthma Network. The Global Asthma Report 2014. http://globalasthmareport.org/index.php. (Auckland, New Zealand, 2014).
Pavord, I. D. et al. After asthma: redefining airways diseases. Lancet. https://doi.org/10.1016/ S0140-6736(17)30879-6 (2017).
Aaron, S. D. et al. Reevaluation of diagnosis in adults with physician-diagnosed asthma. J. Am. Med. Assoc. 317, 269–279 (2017).
José, B. P. D. S., Camargos, P. A. M., Cruz Filho, Á. A. S. D., & Corrêa, R. D. A. Diagnostic accuracy of respiratory diseases in primary health units. Rev. Assoc. Med. Bras. 60, 599–612 (2014).
Looijmans-Van den Akker, I., van Luijn, K. & Verheij, T. Overdiagnosis of asthma in children in primary care: a retrospective analysis. Br. J. Gen. Pract. 66, e152–e157 (2016).
Schneider, A. et al. Diagnostic accuracy of spirometry in primary care. BMC Pulm. Med. 9, 31 (2009).
British Thoracic Society/Scottish Intercollegiate Guideline Network. British Guideline on the Management of Asthma. http://www.sign.ac.uk/sign-153-british- guideline-on-the-management-of-asthma.html (2016).
Global Initiative for Asthma. Global Strategy for Asthma Management and Prevention. http://www.ginasthma.org (2017).
Agusti, A. et al. Treatable traits: toward precision medicine of chronic airway diseases. Eur. Respir. J. 47, 410–419 (2016).
Toll, D. B., Janssen, K. J. M., Vergouwe, Y. & Moons, K. G. M. Validation, updating and impact of clinical prediction rules: a review. J. Clin. Epidemiol. 61, 1085–1094 (2008).
Laupacis, A. & Sekar, N. Clinical prediction rules: a review and suggested modifications of methodological standards. J. Am. Med. Assoc. 277, 488–494 (1997).
Steyerberg, E. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating (Springer Science & Business Media, New York, USA, 2009).
Steyerberg, E. W. et al. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med. 10, e1001381 (2013).
Plüddemann, A. et al. Clinical prediction rules in practice: review of clinical guidelines and survey of GPs. Br. J. Gen. Pract. 64, e233–e242 (2014).
National Institute for Health and Care Excellence: Asthma: Diagnosis, Monitoring and Chronic Asthma Management. https://www.nice.org.uk/guidance/ng80 (2017).
Knottnerus, J. A. Between iatrotropic stimulus and interiatric referral: the domain of primary care research. J. Clin. Epidemiol. 55, 1201–1206 (2002).
Choi, B. W. et al. Easy diagnosis of asthma: computer-assisted, symptom-based diagnosis. J. Korean Med. Sci. 22, 832–838 (2007).
Tomita, K. et al. A scoring algorithm for predicting the presence of adult asthma: a prospective derivation study. Prim. Care Respir. J. 22, 51–58 (2013).
Metting, E. I. et al. Development of a diagnostic decision tree for obstructive pulmonary diseases based on real-life data. ERJ Open Res. 2, 00077–02015 (2016).
Wallace, E. et al. Framework for the impact analysis and implementation of clinical prediction rules (CPRs). BMC Med. Inform. Decis. Mak. 11, 62 (2011).
Wyatt, J. C. & Altman, D. G. Commentary: prognostic models: clinically useful or quickly forgotten? Br. Med. J. 311, 1539–1541 (1995).
Belgrave, D. et al. Disaggregating asthma: big investigation versus big data. J. Allergy Clin. Immunol. 139, 400–407 (2017).
Moons, K. G. et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 11, e1001744 (2014).
Shamseer, L. et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P): elaboration and explanation. Br. Med. J. 349, g7647 (2015).
Ingui, B. J. & Rogers, M. A. Searching for clinical prediction rules in MEDLINE. J. Am. Medical Inform. Assoc. 8, 391–397 (2001).
Geersing, G. J. et al. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS ONE 7 e32844 (2012).
Wolff R, et al. PROBAST: a risk of bias tool for prediction modelling studies. In Challenges to evidence-based health care and Cochrane. Abstracts of the 24th Cochrane Colloquium. https://abstracts.cochrane.org/2016-seoul/probast-risk-bias-tool-prediction-modelling-studies. (John Wiley & Sons, Seoul, Korea 2016).
Debray, T. P. et al. A guide to systematic review and meta-analysis of prediction model performance. Br. Med. J. 356, i6460 (2017).
Begg, C. B. & Mazumdar, M. Operating characteristics of a rank correlation test for publication bias. Biometrics 50, 1088–1101 (1994).
Egger, M., Smith, G. D., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315, 629–634 (1997).
Schünemann, H. J. et al. Rating quality of evidence and strength of recommendations: GRADE: grading quality of evidence and strength of recommendations for diagnostic tests and strategies. Br. Med. J. 336, 1106 (2008).
We are grateful to Marshall Dozier, Academic Librarian at the University of Edinburgh, for her help in developing the search strategy. L.D. is supported by a clinical academic fellowship from Chief Scientist Office, Edinburgh (CAF/17/01). Neither funder nor sponsor (University of Edinburgh) contributed to protocol development.
A.S. is Joint Editor-in-Chief of npj: Primary Care Respiratory Medicine, but was not involved in the editorial review of, nor the decision to publish, this article. The remaining authors declare no competing interests.
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
About this article
Cite this article
Daines, L., McLean, S., Buelo, A. et al. Clinical prediction models to support the diagnosis of asthma in primary care: a systematic review protocol. npj Prim Care Resp Med 28, 15 (2018). https://doi.org/10.1038/s41533-018-0086-6
npj Primary Care Respiratory Medicine (2019)