Protocol | Open | Published:

Clinical prediction models to support the diagnosis of asthma in primary care: a systematic review protocol

npj Primary Care Respiratory Medicinevolume 28, Article number: 15 (2018) | Download Citation

Subjects

Abstract

Substantial over-diagnosis and under-diagnosis of asthma in adults and children has recently been reported. As asthma is mostly diagnosed in non-specialist settings, a clinical prediction model (CPM) to aid the diagnosis of asthma in primary care may help improve diagnostic accuracy. We aim to systematically identify, describe, compare, and synthesise existing CPMs designed to support the diagnosis of asthma in children and adults presenting with symptoms suggestive of the disease, in primary care settings or equivalent populations. We will systematically search Medline, Embase and CINAHL from 1 January 1990 to present. Any CPM derived for use in a primary care population will be included. Equivalent populations in countries without a developed primary care service will also be included. The probability of asthma diagnosis will be the primary outcome. We will include CPMs designed for use in clinical practice to aid the diagnostic decision making of a healthcare professional during the assessment of an individual with symptoms suggestive of asthma. We will include derivation studies, and external model validation studies. Two reviewers will independently screen titles/abstracts and full texts for eligibility and extract data from included papers. The CHARMS checklist (or PROBAST if available) will be used to assess risk of bias within each study. Results will be summarised by narrative synthesis with meta-analyses completed if possible. This systematic review will provide comprehensive information about existing CPMs for the diagnosis of asthma in primary care and will inform the development of a future diagnostic model.

Background

Asthma is a chronic respiratory disease affecting an estimated 334 million people worldwide.1 Progress towards improving asthma outcomes has been limited over the past decade.2 A key determinant in providing optimal asthma care is accurate and timely diagnosis. Yet, asthma remains a challenging disease to diagnose and substantial over-diagnosis and under-diagnosis of asthma in adults and children is reported.3,4,5 The consequences of under-diagnosis are lack of treatment, avoidable morbidity and mortality; while over-diagnosis leads to costly, potentially harmful treatment and unnecessary healthcare use. Furthermore, to realise the potential of precision medicine, greater accuracy in determining asthma traits at the point of diagnosis is required.2

Accurate diagnosis is challenging in a condition defined by its variability, in which the diagnosis is clinical, and in which symptoms, signs, and tests have poor predictive value for the diagnosis.6,7,8 The heterogeneity of asthma and recognition of distinct phenotypes and endotypes, adds further complexity,2,9 particularly as variable airflow obstruction remains the core diagnostic feature of asthma in current guidelines.7,8

Clinical prediction models (CPM), also referred to as prediction rules or risk scores10 combine at least two predictors, such as elements from a clinical history, physical examination, test results or response to treatment, to estimate the probability that a certain outcome is present.10,11 In clinical practice, CPMs can assist healthcare professionals to weigh up the probability of a diagnosis, enhance decision making with patients, or aid patient stratification.12,13,14

The current diagnostic strategies recommended in asthma guidelines are based on expert consensus and in some instances informed by health economic modelling, yet the accuracy with which the advocated clinical assessment pathways result in a correct diagnosis have not been formally evaluated.7,8,15

A prospectively validated CPM to support the diagnosis of asthma would offer an evidence-based approach for the assessment of individuals with symptoms suggestive of asthma. As most asthma diagnoses occur in non-specialist settings,2 a CPM developed for use in primary care (or equivalent settings where undifferentiated health problems are presented to healthcare professionals)16 would be particularly valuable.

To our knowledge, there are few CPMs to support the diagnosis of asthma in primary care. Previous searches identified a symptom-based diagnostic tool from South Korea,17 and an algorithm derived from Japanese healthcare;18 neither have been externally validated. A decision tree used to differentiate obstructive respiratory conditions has been derived and validated in the Netherlands.19 To build on these foundations, inform future development and progress towards wider implementation of a CPM for asthma diagnosis in primary care, a systematic review is needed.

Aims

To identify, describe, compare, and synthesise all CPMs designed to support the diagnosis of asthma in children and adults presenting with symptoms suggestive of the disease, in primary care settings or equivalent populations.

Discussion

Accurately diagnosing asthma is pivotal for effective symptom control and prevention of asthma attacks. By conducting this systematic review of CPMs for the diagnosis of asthma in primary care, we will critically appraise each included model, understand the strengths and limitations of each study design and the utility of each candidate predictor used, including signs and symptoms, personal and family history, and results from past and current clinical tests.

Although a CPM offers potential for improving the diagnosis of asthma, a number of questions need to be addressed before it will be accepted into clinical practice. Of fundamental importance is the outcome measure used to derive the model. In the absence of a “golden” reference standard for asthma diagnosis, the outcome measure must be carefully considered if an accurate diagnosis is to be made. Second, regardless of the statistical accuracy of a model, to be clinically useful, its impact on clinical outcomes should be tested, and healthcare professionals must trust the model and want to use it.20,21 Finally, contemporary understanding of asthma as an aggregate diagnosis2,22 suggests that a future diagnostic CPM may align with a precision medicine approach, potentially being able to stratify individuals by treatable traits.9 Our systematic approach will provide comprehensive information about existing CPMs, and inform the future development of a model for the diagnosis of asthma in primary care.

Methods

The CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) has been used to define the review question, and develop the review design.23 The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guided protocol reporting.24


Data sources and search strategy

We will complete a systematic search to identify all studies describing CPMs to support the diagnosis of asthma in primary care settings. In this review, a CPM is defined as any tool that combines at least two predictors, to provide an estimate for the probability of an outcome.10,14

Medline, Embase, and CINAHL databases will be searched from 1 January 1990 to present. The search strategy (Appendix 1) uses Cochrane asthma search syntax, with established search filters for prediction models.25,26 CPMs published in national clinical guidelines will be sought in TRIP (https://www.tripdatabase.com) and Guidelines Clearinghouse (https://www.guideline.gov). Forward and backward citations from the included studies will be searched for additional eligible studies. International experts and authors known to have published relevant work will be contacted to identify further studies. No language restrictions will be used. All studies will be translated into English where possible, any literature that we are unable to translate will be declared.


Study eligibility criteria

Types of study

We will include derivation studies with or without external validation, in addition to external model validation studies with or without model updating.23

Population

Any CPM derived for use in a primary care population (or equivalent) will be included.

Intervention (type of model)

We will include CPMs designed for use in clinical practice to aid the diagnostic decision making of a healthcare professional during the assessment of an individual with symptoms suggestive of asthma.

Comparator

There are no relevant comparators for this systematic review.

Outcome to be predicted

The primary outcome is the probability of asthma diagnosis. Included studies should present at least one prediction model, or equivalent statistical method, in such a way that allows the probability of asthma to be calculated for other individuals. Some CPMs may use the broader outcome of obstructive airways disease. These studies will be included if data pertaining to asthma diagnosis can be extracted. CPMs derived for the diagnosis of occupational asthma will be included if the intended use is to aid diagnostic decision making in patients with undifferentiated symptoms.

Reference standard used to diagnose asthma

From scoping work, we anticipate that studies designed to derive and validate asthma diagnosis CPMs will use a variety of reference standards. We will evaluate the impact of using different measures as the reference standard. However, studies which do not use an internationally accepted definition for asthma will be excluded.

Exclusion criteria

Studies will be excluded if:

  1. 1.

    The predictive value of more than one variable was evaluated but not combined to produce a diagnostic estimate.

  2. 2.

    Publication occurred before 1 January 1990.

  3. 3.

    Variables used in the model are not available in routine clinical practice or not clearly reported.

  4. 4.

    Outcomes for asthma are not separate or data relating to the asthma outcome is not extractable.

  5. 5.

    The CPM was derived to predict future risk of asthma.

  6. 6.

    Over half of the participants included were children less than 5 years old (because of the overlap between asthma and viral-associated wheeze in this age group).

  7. 7.

    The reference standard used is not based on an internationally recognised definition of asthma.

  8. 8.

    Non-original studies such as editorials, expert views, non-research letters.


Data management and selection process

Records will be retrieved from databases and transferred into a reference manager. Records will be de-duplicated, screened, and managed using Covidence (https://www.covidence.org). Two reviewers (L.D., A.B.) will independently screen titles and abstracts for eligibility. Full-text copies of all potentially relevant records will be obtained. Two reviewers (L.D., A.B. or S.McL.) will independently assess whether each full-text record meets the inclusion criteria. Any discrepancies will be arbitrated by a third reviewer (H.P., S.L., or A.S.).


Data extraction

Included articles will be categorised into two groups: derivation studies and external model validation (with or without model updating). A standardised data extraction form to collect relevant data items from eligible studies will be developed and piloted in line with Cochrane and CHARMS guidance.23 Data extraction will be undertaken by two reviewers (L.D., S.McL.) who will independently extract relevant data from each included study onto the data extraction form. Any disagreements will be resolved by a third reviewer (H.P., S.L., or A.S.).


Data items

Data from the included studies will be summarised in descriptive tables. Table 1 lists the minimum set of data items to be extracted. If a study reports more than one model, we will extract data separately for each model.

Table 1 Minimum data items to be extracted from studies reporting model derivation or model validation

Critical appraisal and risk of bias in individual studies

The CHARMS checklist will guide risk of bias assessment within individual studies.23 The prediction model risk of bias assessment tool (PROBAST) will be used if available.27 Quality assessment will be used to evaluate whether study quality makes any impression on the conclusions.


Data synthesis

Results will be summarised by narrative synthesis and descriptive statistics. Methods for meta-analysis of model validation studies have recently been reported28 and will be considered if suitable studies are identified.


Publication bias

If possible, we will evaluate publication bias using Begg and Egger tests and funnel plots.29,30


Evaluating confidence in cumulative evidence

We will evaluate the overall evidence and strength of recommendations gathered from the review using the Grading of Recommendations Assessment, Development and Evaluation system.31


Protocol registration

A protocol will be registered with the International Prospective Register of Systematic Reviews (PROSPERO). In the event of a protocol amendment, the date, description, and rationale will be reported.


Data availability

Data sharing is not applicable as this article describes a protocol and no datasets were generated or analysed.

Additional information

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Global Asthma Network. The Global Asthma Report 2014. http://globalasthmareport.org/index.php. (Auckland, New Zealand, 2014).

  2. 2.

    Pavord, I. D. et al. After asthma: redefining airways diseases. Lancet. https://doi.org/10.1016/ S0140-6736(17)30879-6 (2017).

  3. 3.

    Aaron, S. D. et al. Reevaluation of diagnosis in adults with physician-diagnosed asthma. J. Am. Med. Assoc. 317, 269–279 (2017).

  4. 4.

    José, B. P. D. S., Camargos, P. A. M., Cruz Filho, Á. A. S. D., & Corrêa, R. D. A. Diagnostic accuracy of respiratory diseases in primary health units. Rev. Assoc. Med. Bras. 60, 599–612 (2014).

  5. 5.

    Looijmans-Van den Akker, I., van Luijn, K. & Verheij, T. Overdiagnosis of asthma in children in primary care: a retrospective analysis. Br. J. Gen. Pract. 66, e152–e157 (2016).

  6. 6.

    Schneider, A. et al. Diagnostic accuracy of spirometry in primary care. BMC Pulm. Med. 9, 31 (2009).

  7. 7.

    British Thoracic Society/Scottish Intercollegiate Guideline Network. British Guideline on the Management of Asthma. http://www.sign.ac.uk/sign-153-british- guideline-on-the-management-of-asthma.html (2016).

  8. 8.

    Global Initiative for Asthma. Global Strategy for Asthma Management and Prevention. http://www.ginasthma.org (2017).

  9. 9.

    Agusti, A. et al. Treatable traits: toward precision medicine of chronic airway diseases. Eur. Respir. J. 47, 410–419 (2016).

  10. 10.

    Toll, D. B., Janssen, K. J. M., Vergouwe, Y. & Moons, K. G. M. Validation, updating and impact of clinical prediction rules: a review. J. Clin. Epidemiol. 61, 1085–1094 (2008).

  11. 11.

    Laupacis, A. & Sekar, N. Clinical prediction rules: a review and suggested modifications of methodological standards. J. Am. Med. Assoc. 277, 488–494 (1997).

  12. 12.

    Steyerberg, E. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating (Springer Science & Business Media, New York, USA, 2009).

  13. 13.

    Steyerberg, E. W. et al. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med. 10, e1001381 (2013).

  14. 14.

    Plüddemann, A. et al. Clinical prediction rules in practice: review of clinical guidelines and survey of GPs. Br. J. Gen. Pract. 64, e233–e242 (2014).

  15. 15.

    National Institute for Health and Care Excellence: Asthma: Diagnosis, Monitoring and Chronic Asthma Management. https://www.nice.org.uk/guidance/ng80 (2017).

  16. 16.

    Knottnerus, J. A. Between iatrotropic stimulus and interiatric referral: the domain of primary care research. J. Clin. Epidemiol. 55, 1201–1206 (2002).

  17. 17.

    Choi, B. W. et al. Easy diagnosis of asthma: computer-assisted, symptom-based diagnosis. J. Korean Med. Sci. 22, 832–838 (2007).

  18. 18.

    Tomita, K. et al. A scoring algorithm for predicting the presence of adult asthma: a prospective derivation study. Prim. Care Respir. J. 22, 51–58 (2013).

  19. 19.

    Metting, E. I. et al. Development of a diagnostic decision tree for obstructive pulmonary diseases based on real-life data. ERJ Open Res. 2, 00077–02015 (2016).

  20. 20.

    Wallace, E. et al. Framework for the impact analysis and implementation of clinical prediction rules (CPRs). BMC Med. Inform. Decis. Mak. 11, 62 (2011).

  21. 21.

    Wyatt, J. C. & Altman, D. G. Commentary: prognostic models: clinically useful or quickly forgotten? Br. Med. J. 311, 1539–1541 (1995).

  22. 22.

    Belgrave, D. et al. Disaggregating asthma: big investigation versus big data. J. Allergy Clin. Immunol. 139, 400–407 (2017).

  23. 23.

    Moons, K. G. et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. 11, e1001744 (2014).

  24. 24.

    Shamseer, L. et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P): elaboration and explanation. Br. Med. J. 349, g7647 (2015).

  25. 25.

    Ingui, B. J. & Rogers, M. A. Searching for clinical prediction rules in MEDLINE. J. Am. Medical Inform. Assoc. 8, 391–397 (2001).

  26. 26.

    Geersing, G. J. et al. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLoS ONE 7 e32844 (2012).

  27. 27.

    Wolff R, et al. PROBAST: a risk of bias tool for prediction modelling studies. In Challenges to evidence-based health care and Cochrane. Abstracts of the 24th Cochrane Colloquium. https://abstracts.cochrane.org/2016-seoul/probast-risk-bias-tool-prediction-modelling-studies. (John Wiley & Sons, Seoul, Korea 2016).

  28. 28.

    Debray, T. P. et al. A guide to systematic review and meta-analysis of prediction model performance. Br. Med. J. 356, i6460 (2017).

  29. 29.

    Begg, C. B. & Mazumdar, M. Operating characteristics of a rank correlation test for publication bias. Biometrics 50, 1088–1101 (1994).

  30. 30.

    Egger, M., Smith, G. D., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315, 629–634 (1997).

  31. 31.

    Schünemann, H. J. et al. Rating quality of evidence and strength of recommendations: GRADE: grading quality of evidence and strength of recommendations for diagnostic tests and strategies. Br. Med. J. 336, 1106 (2008).

Download references

Acknowledgements

We are grateful to Marshall Dozier, Academic Librarian at the University of Edinburgh, for her help in developing the search strategy. L.D. is supported by a clinical academic fellowship from Chief Scientist Office, Edinburgh (CAF/17/01). Neither funder nor sponsor (University of Edinburgh) contributed to protocol development.

Author information

Affiliations

  1. Asthma UK Centre for Applied Research, Usher Institute of Population Health Sciences and Informatics, The University of Edinburgh, Edinburgh, Scotland

    • L. Daines
    • , S. McLean
    • , A. Sheikh
    •  & H. Pinnock
  2. Scottish Collaboration for Public Health Research and Policy, The University of Edinburgh, Edinburgh, Scotland

    • A. Buelo
  3. Edinburgh Clinical Trials Unit, Usher Institute of Population Health Sciences and Informatics, The University of Edinburgh, Edinburgh, Scotland

    • S. Lewis

Authors

  1. Search for L. Daines in:

  2. Search for S. McLean in:

  3. Search for A. Buelo in:

  4. Search for S. Lewis in:

  5. Search for A. Sheikh in:

  6. Search for H. Pinnock in:

Contributions

L.D. and H.P. conceived the idea for this work supported by A.S. and S.L. L.D. wrote the first draft, and all authors contributed to the manuscript.

Competing interests

A.S. is Joint Editor-in-Chief of npj: Primary Care Respiratory Medicine, but was not involved in the editorial review of, nor the decision to publish, this article. The remaining authors declare no competing interests.

Corresponding author

Correspondence to L. Daines.

Electronic supplementary material

About this article

Publication history

Received

Revised

Accepted

Published

DOI

https://doi.org/10.1038/s41533-018-0086-6

Rights and permissions

Creative Commons BY

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.