Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Clinical prediction models to support the diagnosis of asthma in primary care: a systematic review protocol

Subjects

Abstract

Substantial over-diagnosis and under-diagnosis of asthma in adults and children has recently been reported. As asthma is mostly diagnosed in non-specialist settings, a clinical prediction model (CPM) to aid the diagnosis of asthma in primary care may help improve diagnostic accuracy. We aim to systematically identify, describe, compare, and synthesise existing CPMs designed to support the diagnosis of asthma in children and adults presenting with symptoms suggestive of the disease, in primary care settings or equivalent populations. We will systematically search Medline, Embase and CINAHL from 1 January 1990 to present. Any CPM derived for use in a primary care population will be included. Equivalent populations in countries without a developed primary care service will also be included. The probability of asthma diagnosis will be the primary outcome. We will include CPMs designed for use in clinical practice to aid the diagnostic decision making of a healthcare professional during the assessment of an individual with symptoms suggestive of asthma. We will include derivation studies, and external model validation studies. Two reviewers will independently screen titles/abstracts and full texts for eligibility and extract data from included papers. The CHARMS checklist (or PROBAST if available) will be used to assess risk of bias within each study. Results will be summarised by narrative synthesis with meta-analyses completed if possible. This systematic review will provide comprehensive information about existing CPMs for the diagnosis of asthma in primary care and will inform the development of a future diagnostic model.

Background

Asthma is a chronic respiratory disease affecting an estimated 334 million people worldwide.1 Progress towards improving asthma outcomes has been limited over the past decade.2 A key determinant in providing optimal asthma care is accurate and timely diagnosis. Yet, asthma remains a challenging disease to diagnose and substantial over-diagnosis and under-diagnosis of asthma in adults and children is reported.3,4,5 The consequences of under-diagnosis are lack of treatment, avoidable morbidity and mortality; while over-diagnosis leads to costly, potentially harmful treatment and unnecessary healthcare use. Furthermore, to realise the potential of precision medicine, greater accuracy in determining asthma traits at the point of diagnosis is required.2

Accurate diagnosis is challenging in a condition defined by its variability, in which the diagnosis is clinical, and in which symptoms, signs, and tests have poor predictive value for the diagnosis.6,7,8 The heterogeneity of asthma and recognition of distinct phenotypes and endotypes, adds further complexity,2,9 particularly as variable airflow obstruction remains the core diagnostic feature of asthma in current guidelines.7,8

Clinical prediction models (CPM), also referred to as prediction rules or risk scores10 combine at least two predictors, such as elements from a clinical history, physical examination, test results or response to treatment, to estimate the probability that a certain outcome is present.10,11 In clinical practice, CPMs can assist healthcare professionals to weigh up the probability of a diagnosis, enhance decision making with patients, or aid patient stratification.12,13,14

The current diagnostic strategies recommended in asthma guidelines are based on expert consensus and in some instances informed by health economic modelling, yet the accuracy with which the advocated clinical assessment pathways result in a correct diagnosis have not been formally evaluated.7,8,15

A prospectively validated CPM to support the diagnosis of asthma would offer an evidence-based approach for the assessment of individuals with symptoms suggestive of asthma. As most asthma diagnoses occur in non-specialist settings,2 a CPM developed for use in primary care (or equivalent settings where undifferentiated health problems are presented to healthcare professionals)16 would be particularly valuable.

To our knowledge, there are few CPMs to support the diagnosis of asthma in primary care. Previous searches identified a symptom-based diagnostic tool from South Korea,17 and an algorithm derived from Japanese healthcare;18 neither have been externally validated. A decision tree used to differentiate obstructive respiratory conditions has been derived and validated in the Netherlands.19 To build on these foundations, inform future development and progress towards wider implementation of a CPM for asthma diagnosis in primary care, a systematic review is needed.

Aims

To identify, describe, compare, and synthesise all CPMs designed to support the diagnosis of asthma in children and adults presenting with symptoms suggestive of the disease, in primary care settings or equivalent populations.

Discussion

Accurately diagnosing asthma is pivotal for effective symptom control and prevention of asthma attacks. By conducting this systematic review of CPMs for the diagnosis of asthma in primary care, we will critically appraise each included model, understand the strengths and limitations of each study design and the utility of each candidate predictor used, including signs and symptoms, personal and family history, and results from past and current clinical tests.

Although a CPM offers potential for improving the diagnosis of asthma, a number of questions need to be addressed before it will be accepted into clinical practice. Of fundamental importance is the outcome measure used to derive the model. In the absence of a “golden” reference standard for asthma diagnosis, the outcome measure must be carefully considered if an accurate diagnosis is to be made. Second, regardless of the statistical accuracy of a model, to be clinically useful, its impact on clinical outcomes should be tested, and healthcare professionals must trust the model and want to use it.20,21 Finally, contemporary understanding of asthma as an aggregate diagnosis2,22 suggests that a future diagnostic CPM may align with a precision medicine approach, potentially being able to stratify individuals by treatable traits.9 Our systematic approach will provide comprehensive information about existing CPMs, and inform the future development of a model for the diagnosis of asthma in primary care.

Methods

The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) has been used to define the review question, and develop the review design.23 The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guided protocol reporting.24

Data sources and search strategy

We will complete a systematic search to identify all studies describing CPMs to support the diagnosis of asthma in primary care settings. In this review, a CPM is defined as any tool that combines at least two predictors, to provide an estimate for the probability of an outcome.10,14

Medline, Embase, and CINAHL databases will be searched from 1 January 1990 to present. The search strategy (Appendix 1) uses Cochrane asthma search syntax, with established search filters for prediction models.25,26 CPMs published in national clinical guidelines will be sought in TRIP (https://www.tripdatabase.com) and Guidelines Clearinghouse (https://www.guideline.gov). Forward and backward citations from the included studies will be searched for additional eligible studies. International experts and authors known to have published relevant work will be contacted to identify further studies. No language restrictions will be used. All studies will be translated into English where possible, any literature that we are unable to translate will be declared.

Study eligibility criteria

Types of study

We will include derivation studies with or without external validation, in addition to external model validation studies with or without model updating.23

Population

Any CPM derived for use in a primary care population (or equivalent) will be included.

Intervention (type of model)

We will include CPMs designed for use in clinical practice to aid the diagnostic decision making of a healthcare professional during the assessment of an individual with symptoms suggestive of asthma.

Comparator

There are no relevant comparators for this systematic review.

Outcome to be predicted

The primary outcome is the probability of asthma diagnosis. Included studies should present at least one prediction model, or equivalent statistical method, in such a way that allows the probability of asthma to be calculated for other individuals. Some CPMs may use the broader outcome of obstructive airways disease. These studies will be included if data pertaining to asthma diagnosis can be extracted. CPMs derived for the diagnosis of occupational asthma will be included if the intended use is to aid diagnostic decision making in patients with undifferentiated symptoms.

Reference standard used to diagnose asthma

From scoping work, we anticipate that studies designed to derive and validate asthma diagnosis CPMs will use a variety of reference standards. We will evaluate the impact of using different measures as the reference standard. However, studies which do not use an internationally accepted definition for asthma will be excluded.

Exclusion criteria

Studies will be excluded if:

  1. 1.

    The predictive value of more than one variable was evaluated but not combined to produce a diagnostic estimate.

  2. 2.

    Publication occurred before 1 January 1990.

  3. 3.

    Variables used in the model are not available in routine clinical practice or not clearly reported.

  4. 4.

    Outcomes for asthma are not separate or data relating to the asthma outcome is not extractable.

  5. 5.

    The CPM was derived to predict future risk of asthma.

  6. 6.

    Over half of the participants included were children less than 5 years old (because of the overlap between asthma and viral-associated wheeze in this age group).

  7. 7.

    The reference standard used is not based on an internationally recognised definition of asthma.

  8. 8.

    Non-original studies such as editorials, expert views, non-research letters.

Data management and selection process

Records will be retrieved from databases and transferred into a reference manager. Records will be de-duplicated, screened, and managed using Covidence (https://www.covidence.org). Two reviewers (L.D., A.B.) will independently screen titles and abstracts for eligibility. Full-text copies of all potentially relevant records will be obtained. Two reviewers (L.D., A.B. or S.McL.) will independently assess whether each full-text record meets the inclusion criteria. Any discrepancies will be arbitrated by a third reviewer (H.P., S.L., or A.S.).

Data extraction

Included articles will be categorised into two groups: derivation studies and external model validation (with or without model updating). A standardised data extraction form to collect relevant data items from eligible studies will be developed and piloted in line with Cochrane and CHARMS guidance.23 Data extraction will be undertaken by two reviewers (L.D., S.McL.) who will independently extract relevant data from each included study onto the data extraction form. Any disagreements will be resolved by a third reviewer (H.P., S.L., or A.S.).

Data items

Data from the included studies will be summarised in descriptive tables. Table 1 lists the minimum set of data items to be extracted. If a study reports more than one model, we will extract data separately for each model.

Table 1 Minimum data items to be extracted from studies reporting model derivation or model validation

Critical appraisal and risk of bias in individual studies

The CHARMS checklist will guide risk of bias assessment within individual studies.23 The prediction model risk of bias assessment tool (PROBAST) will be used if available.27 Quality assessment will be used to evaluate whether study quality makes any impression on the conclusions.

Data synthesis

Results will be summarised by narrative synthesis and descriptive statistics. Methods for meta-analysis of model validation studies have recently been reported28 and will be considered if suitable studies are identified.

Publication bias

If possible, we will evaluate publication bias using Begg and Egger tests and funnel plots.29,30

Evaluating confidence in cumulative evidence

We will evaluate the overall evidence and strength of recommendations gathered from the review using the Grading of Recommendations Assessment, Development and Evaluation system.31

Protocol registration

A protocol will be registered with the International Prospective Register of Systematic Reviews (PROSPERO). In the event of a protocol amendment, the date, description, and rationale will be reported.

Data availability

Data sharing is not applicable as this article describes a protocol and no datasets were generated or analysed.

References

  1. 1.

    Global Asthma Network. The Global Asthma Report 2014. http://globalasthmareport.org/index.php. (Auckland, New Zealand, 2014).

  2. 2.

    Pavord, I. D. et al. After asthma: redefining airways diseases. Lancet. https://doi.org/10.1016/ S0140-6736(17)30879-6 (2017).

  3. 3.

    Aaron, S. D. et al. Reevaluation of diagnosis in adults with physician-diagnosed asthma. J. Am. Med. Assoc. 317, 269–279 (2017).

    Article  Google Scholar 

  4. 4.

    José, B. P. D. S., Camargos, P. A. M., Cruz Filho, Á. A. S. D., & Corrêa, R. D. A. Diagnostic accuracy of respiratory diseases in primary health units. Rev. Assoc. Med. Bras. 60, 599–612 (2014).

  5. 5.

    Looijmans-Van den Akker, I., van Luijn, K. & Verheij, T. Overdiagnosis of asthma in children in primary care: a retrospective analysis. Br. J. Gen. Pract. 66, e152–e157 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Schneider, A. et al. Diagnostic accuracy of spirometry in primary care. BMC Pulm. Med. 9, 31 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    British Thoracic Society/Scottish Intercollegiate Guideline Network. British Guideline on the Management of Asthma. http://www.sign.ac.uk/sign-153-british- guideline-on-the-management-of-asthma.html (2016).

  8. 8.

    Global Initiative for Asthma. Global Strategy for Asthma Management and Prevention. http://www.ginasthma.org (2017).

  9. 9.

    Agusti, A. et al. Treatable traits: toward precision medicine of chronic airway diseases. Eur. Respir. J. 47, 410–419 (2016).

    Article  PubMed  Google Scholar 

  10. 10.

    Toll, D. B., Janssen, K. J. M., Vergouwe, Y. & Moons, K. G. M. Validation, updating and impact of clinical prediction rules: a review. J. Clin. Epidemiol. 61, 1085–1094 (2008).

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    Laupacis, A. & Sekar, N. Clinical prediction rules: a review and suggested modifications of methodological standards. J. Am. Med. Assoc. 277, 488–494 (1997).

    CAS  Article  Google Scholar 

  12. 12.

    Steyerberg, E. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating (Springer Science & Business Media, New York, USA, 2009).

  13. 13.

    Steyerberg, E. W. et al. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med. 10, e1001381 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Plüddemann, A. et al. Clinical prediction rules in practice: review of clinical guidelines and survey of GPs. Br. J. Gen. Pract. 64, e233–e242 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  15. 15.

    National Institute for Health and Care Excellence: Asthma: Diagnosis, Monitoring and Chronic Asthma Management. https://www.nice.org.uk/guidance/ng80 (2017).

  16. 16.

    Knottnerus, J. A. Between iatrotropic stimulus and interiatric referral: the domain of primary care research. J. Clin. Epidemiol. 55, 1201–1206 (2002).

    Article  PubMed  Google Scholar 

  17. 17.

    Choi, B. W. et al. Easy diagnosis of asthma: computer-assisted, symptom-based diagnosis. J. Korean Med. Sci. 22, 832–838 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  18. 18.

    Tomita, K. et al. A scoring algorithm for predicting the presence of adult asthma: a prospective derivation study. Prim. Care Respir. J. 22, 51–58 (2013).

    Article  PubMed  Google Scholar 

  19. 19.

    Metting, E. I. et al. Development of a diagnostic decision tree for obstructive pulmonary diseases based on real-life data. ERJ Open Res. 2, 00077–02015 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Wallace, E. et al. Framework for the impact analysis and implementation of clinical prediction rules (CPRs). BMC Med. Inform. Decis. Mak. 11, 62 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  21. 21.

    Wyatt, J. C. & Altman, D. G. Commentary: prognostic models: clinically useful or quickly forgotten? Br. Med. J. 311, 1539–1541 (1995).

    Article  Google Scholar 

  22. 22.

    Belgrave, D. et al. Disaggregating asthma: big investigation versus big data. J. Allergy Clin. Immunol. 139, 400–407 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Moons, K. G. et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 11, e1001744 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  24. 24.

    Shamseer, L. et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P): elaboration and explanation. Br. Med. J. 349, g7647 (2015).

    Article  Google Scholar 

  25. 25.

    Ingui, B. J. & Rogers, M. A. Searching for clinical prediction rules in MEDLINE. J. Am. Medical Inform. Assoc. 8, 391–397 (2001).

    CAS  Article  Google Scholar 

  26. 26.

    Geersing, G. J. et al. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS ONE 7 e32844 (2012).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  27. 27.

    Wolff R, et al. PROBAST: a risk of bias tool for prediction modelling studies. In Challenges to evidence-based health care and Cochrane. Abstracts of the 24th Cochrane Colloquium. https://abstracts.cochrane.org/2016-seoul/probast-risk-bias-tool-prediction-modelling-studies. (John Wiley & Sons, Seoul, Korea 2016).

  28. 28.

    Debray, T. P. et al. A guide to systematic review and meta-analysis of prediction model performance. Br. Med. J. 356, i6460 (2017).

    Article  Google Scholar 

  29. 29.

    Begg, C. B. & Mazumdar, M. Operating characteristics of a rank correlation test for publication bias. Biometrics 50, 1088–1101 (1994).

    CAS  Article  PubMed  Google Scholar 

  30. 30.

    Egger, M., Smith, G. D., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315, 629–634 (1997).

    CAS  Article  Google Scholar 

  31. 31.

    Schünemann, H. J. et al. Rating quality of evidence and strength of recommendations: GRADE: grading quality of evidence and strength of recommendations for diagnostic tests and strategies. Br. Med. J. 336, 1106 (2008).

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to Marshall Dozier, Academic Librarian at the University of Edinburgh, for her help in developing the search strategy. L.D. is supported by a clinical academic fellowship from Chief Scientist Office, Edinburgh (CAF/17/01). Neither funder nor sponsor (University of Edinburgh) contributed to protocol development.

Author information

Affiliations

Authors

Contributions

L.D. and H.P. conceived the idea for this work supported by A.S. and S.L. L.D. wrote the first draft, and all authors contributed to the manuscript.

Corresponding author

Correspondence to L. Daines.

Ethics declarations

Competing interests

A.S. is Joint Editor-in-Chief of npj: Primary Care Respiratory Medicine, but was not involved in the editorial review of, nor the decision to publish, this article. The remaining authors declare no competing interests.

Additional information

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Daines, L., McLean, S., Buelo, A. et al. Clinical prediction models to support the diagnosis of asthma in primary care: a systematic review protocol. npj Prim Care Resp Med 28, 15 (2018). https://doi.org/10.1038/s41533-018-0086-6

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing