Research Paper

Predicting asthma in preschool children at high risk presenting in primary care: development of a clinical asthma prediction score

  • Primary Care Respiratory Journal (2014) 23, 5259
  • doi:10.4104/pcrj.2014.00003
  • Download Citation
Received:
Revised:
Accepted:
Published online:

Abstract

Background:

A setting-specific asthma prediction score for preschool children with wheezing and/or dyspnoea presenting in primary healthcare is needed since existing indices are mainly based on general populations.

Aims:

To find an optimally informative yet practical set of predictors for the prediction of asthma in preschool children at high risk who present in primary healthcare.

Methods:

A total of 771 Dutch preschool children at high risk of asthma were followed prospectively until the age of six years. Data on asthma symptoms and environmental conditions were obtained using validated questionnaires and specific IgE was measured. At the age of six years the presence of asthma was assessed based on asthma symptoms, medication, and bronchial hyper-responsiveness. A clinical asthma prediction score (CAPS) was developed using bootstrapped multivariable regression methods.

Results:

In all, 438 children (56.8%) completed the study; the asthma prevalence at six years was 42.7%. Five parameters optimally predicted asthma: age, family history of asthma or allergy, wheezing-induced sleep disturbances, wheezing in the absence of common colds, and specific IgE. CAPS scores range from 0 to 11 points; scores <3 signified a negative predictive value of 78.4% while scores of ≥7 signified a positive predictive value of 74.3%.

Conclusions:

We have developed an easy-to-use CAPS for preschool children with symptoms suggesting asthma who present in primary healthcare. After suitable validation, the CAPS may assist in guiding shared decision-making to tailor the need for medical or non-medical interventions. External validation of the CAPS is needed.

Introduction

Childhood asthma usually starts during the preschool years. Unfortunately, it is currently not possible to test for or alter the natural history of asthma accurately in young children,1,2 so asthma during the preschool years is frequently underdiagnosed. About 10 asthma risk prediction indices for young children have been developed, although only one was developed in a primary care setting where most of these children present with respiratory symptoms for the first time and where most are treated. Risk stratification using prediction indices may be useful for clinical and research purposes. For example, risk stratification supports the tailoring of treatment strategies such that benefits and harms may be properly weighed.3 It may therefore improve the follow-up of children at risk by decreasing the disease burden and improving management, resulting in fewer respiratory symptoms, exacerbations, and emergency medical visits.4,5

Prediction indices are valuable because the information itself may contribute to patients' quality of life.6 Patients' emotional response to medical testing is a natural effect. The information involved in a high predicted probability of a young child having asthma at six years of age may enable them (or their parent/guardians) to make sense of the symptoms or to adjust their lifestyle and living conditions in light of the results, while a low predicted probability may alleviate their concerns.6,7

In addition, various research groups are investigating the underlying pathogenic mechanisms and preventive strategies for asthma,8 and as soon as effective therapy that is either costly and/or has adverse effects emerges, risk stratification will be useful.3

Existing predictive indices have an uncertain value for the primary healthcare setting because the majority of the cohort studies were based on parental questionnaires in a general population,9,​10,​11,​12,​13 so they are of modest clinical value for preschool children.14 For example, the most well-known indices — the loose and stringent Asthma Predictive Indices (API) — were developed in a general population and are based on questionnaires.9 API predicts persistent wheezing in children who ever wheezed and does not predict asthma in children with recurrent wheeze presenting to a physician.15

In 2004 we started a prospective cohort of young children at high risk of developing asthma. The objective was to construct a clinical index for predicting asthma in preschool children at risk of asthma presenting in primary care. Here we describe the development and performance of a novel clinical asthma prediction score (CAPS).

Methods

AppendixDesign and subjects

The subjects were children aged one to five years in the Netherlands who were participating in the ARCADE (AiRway Complaints and Asthma Development) prospective cohort study.4 They had presented at one of the pertinent 14 primary care clinics in the previous 12 months with complaints of recurrent coughing (≥2 visits), wheezing (≥1 visit), and/or shortness of breath (≥1 visit). The study was approved by the Netherlands Central Committee on Research Involving Human Subjects (CCMO/P04.0098C) and the details of the study have been published elsewhere.4 Written informed consent was obtained from the parents before all measurements.

Data collection

Questionnaires

For this study we derived demographic data and data with respect to the child's clinical history from questionnaires obtained by the Core Questionnaire of the International Study of Asthma and Allergy in Children (ISAAC), which was administered at baseline and at the age of six years.

Allergy test

Total immunoglobulin E (IgE) and specific IgE directed against cat, dog, and house dust mite were determined by radioallergosorbent test (RAST) in all the children at baseline. IgE positivity to cat, dog, and house dust mite was defined as >0.35kU/L.4

Spirometry and bronchial hyper-responsiveness

At six years of age, spirometry and bronchial hyper-responsiveness (BHR) assessments were obtained in children with wheezing, shortness of breath, recurrent coughing, or use of asthma medication (β2-agonists, inhaled corticosteroids) during the previous 12 months. BHR to increasing doses of methacholine was measured according to Cockcroft et al.(online supplement reference 3).

Outcome: asthma diagnosis at age six

The diagnosis of asthma at six years of age was defined as: having persistent symptoms of asthma and/or using asthma medication in the last year in combination with BHR (defined as the methacholine dose (<8.0mg/ml) causing a 20% reduction in forced expiratory volume in one second (PC20-FEV1)) or reversibility (>10% increase in FEV1 after short-acting β2-agonists).4 Persistent symptoms were defined as self-reported complaints during the previous 12 months of wheezing and/or shortness of breath and/or recurrent coughing.

Children who had not experienced asthma-related symptoms in the previous year or who had not used asthma medication in the previous year were not invited for lung function measurement. They were considered as not having asthma.

Probability of asthma at the age of six

Depending on the direction and magnitude of the selected predictors as measured by regression coefficients, we developed the Clinical Asthma Prediction Score (CAPS) as a points system which is easy to use, without requiring a computer or calculator.16 To use the CAPS in clinical practice, thresholds are needed to either rule in or rule out asthma at the age of six years. Therefore we set two thresholds — a lower limit at 30% below which we assumed that many physicians may well choose a ‘wait and see’ policy, and an upper limit at 60% above which it was assumed that physicians may pursue a more active management such as a trial of asthma treatment. These thresholds imply that children with a predicted risk of asthma risk between 30% and 60% will be assigned to the area of clinical indecision in which a ‘watchful waiting’ policy may be defensible.

Statistical analysis

Details of the statistical analysis are described in Appendix 1, available online at www.thepcrj.org. Briefly, we analysed whether dropout was selective and missing predictor data were imputed using multiple imputation.17 Twelve candidate predictors of all available data were preselected based on clinical expert opinion after removing five due to collinearity (Table 1). Bootstrapped (1000x) backward selection was used to select the final important predictors whose regression coefficients were shrunk to facilitate better performance when the model is applied to other data or future patients.18 The predictive performance of the prediction index was assessed by its calibration and discrimination abilities.

Table 1: Candidate predictors of asthma diagnosed at 6 years of age in preschool children

Furthermore, the additional value of specific IgE was calculated by the additional area under the ROC curve (AUCdiff) and the net reclassification improvement (NRI).19 All statistical analyses were carried out in Stata/SE Version 10.1 (Stata Corp, College Station, TX, USA).

Results

Of the 771 children, 438 (56.8%) had complete information at baseline and an asthma diagnosis at the endpoint (Figure 1). Of these, 187 (42.7%) had asthma at the age of six. Table 1 shows the univariable frequencies of the preselected candidate predictors. The mean (SD) age at baseline was 3.00 (1.29) years and 189 (43.1%) were female. In 156 children (35.6%), asthma was present in either the parents or siblings. Missingness analyses showed no strong indications for selective drop out (Appendix 1 available online at www.pcrj.org).

Figure 1: Flow diagram
Figure 1

Multivariable analysis

Five predictors were included (Table 2): age, family history of asthma or allergy, wheezing-induced sleep disturbances in the previous 12 months, wheezing in the absence of a cold in the previous 12 months, and specific IgE test. The discrimination of the model expressed as area under the ROC curve was 0.73 (SE=0.024). Calibration was satisfactory (Table 2).

Table 2: Predictors for asthma at age of six years and prediction model performance measures

IgE and no IgE in the model

The model performance was calculated for the model including age, family history, wheezing-induced sleep disturbances, and wheezing in the absence of a cold but excluding IgE. The discrimination of the model expressed as area under the ROC curve was 0.71 (SE=0.024). Calibration was good (Table 2). Adding specific IgE to the model increased the AUC by 2.8 percentage points (AUCdiff= 0.028, 95% CI 0.0047 to 0.051, SE=0.012). Net reclassification analysis showed that 11.23% (p=0.09) of the children were correctly reclassified when IgE was added to the model based on history items only (Table 3).19

Table 3: Reclassification tables for model with or without IgE in children with and without asthma

Probability of asthma at the age of six using CAPS

A points system ranging from 0 to 11 was developed. Thresholds to accept or reject the presence or development of asthma were set at 30% and 60%, respectively (Table 4).16 The positive predictive values (PPVs) of the CAPS (≥7=probability of asthma of ≥60%) were 74.3% (CAPS=7) and 100% (CAPS=11) and the negative predictive values (NPVs) of the CAPS (<3=probability of asthma of ≤30%) were 78.4% (CAPS=2) and 88.2% (CAPS=0) (see online supplement). Figure 2 shows asthma risks as a function of CAPS scores.

Table 4: Risk estimates associated with Clinical Asthma Prediction Score (CAPS) points
Figure 2: Probability of having asthma at age 6 years, including 95% confidence internvals, associated with CAPS
Figure 2

Discussion

Main findings

The clinical asthma prediction score (CAPS) we developed in a prospective cohort study facilitates the risk stratification of preschool children who attend primary healthcare with wheezing, dyspnoea, or recurrent coughing. The predictive ability of the CAPS may assist physicians to distinguish between a relatively low risk (14%) and a very high risk (90%) of developing asthma. The CAPS uses easily accessible information on age, family history of asthma or allergy, wheezing-induced sleep disturbances, and wheezing in the absence of a cold, and an additional blood sample to obtain specific IgE.

The use of the CAPS may enable physicians to reduce the number of children unnecessarily labelled with asthma (low CAPS). For this group, the prior probability went from 42.7% to 21.6%; this helps to reduce unnecessary testing, treatment, and uncertainty. On the other hand, CAPS may allow physicians to identify preschool children who may need proper asthma management (high CAPS). For this group, the prior probability went from 42.7% to 74.3%; a high CAPS may therefore reduce the burden of asthma in these children through better symptom control and fewer exacerbations and hospital admissions. In both situations there will be a reduction in costs. Children with an intermediate CAPS (3>7) retain a probability similar to the probability before applying the CAPS. In this group a watchful waiting policy seems justified, in line with the British guideline,20 which subsequently suggests a watchful waiting approach with review in the majority of cases with intermediate risk predictions.

Strengths and limitations of this study

Our study has the following strengths. First, we selected children from a relevant population — namely, those children in whom an attending primary care physician had considered asthma. Second, all predictors collected are easily accessible in primary care. Third, we used a stringent definition of asthma to ensure a reliable reference standard diagnosis for research purposes. Fourth, we followed the relevant STARD checklist criteria — a strict protocol with well-defined inclusion and exclusion criteria for the setting and population and a generally accepted reference standard.21 Fifth, shrinkage of the magnitude of the regression coefficients was used to counteract over-optimism of our model, thus enhancing its validity outside our data set.17,18 Sixth, we used NRI to measure the additional value of specific IgE to the model. NRI is a marker to express the model performance and to compare models, focusing on the correct reclassification of patients with and without asthma; it appears to be a clinically more sensible approach than the differences between AUCs.

We see the following limitations to our work. First, our study is a model development study without external validation. We did, however, use bootstrapped variable selection and shrinkage of regression coefficients to temper an over-optimistic performance of CAPS in external populations. Second, we have a high dropout rate. Missingness analyses showed that dropout was higher in children of parents who smoked (OR=1.88) and those with persistent (>1 month) coughing (OR=1.62). Parental smoking may be explained as a psychological cause for dropout; parents who smoke may feel guilty or embarrassed that smoking might have a negative influence on their child's physical condition. We assumed that persistent coughing as a cause of dropout was a coincidental finding. The high dropout rate may also be explained by the fact that, due to an extension of follow-up, participants had to explicitly renew their informed consent. Third, although some combinations of predictor values yield low (<30%) or high (>60%) asthma probabilities, almost 58% of the children in our study population had a predicted probability between 30% and 60%, close to the overall prevalence of 42%. For such children, CAPS results are not likely to change clinical management. Finally, the choice of different thresholds may lead to different results. As far as we are aware, there is no evidence-based advice on thresholds for this type of decision-making in the asthma literature. We decided to use thresholds as a clinically intuitive alternative to more formal methods of reclassification and as an attractive alternative for classical methods of predictive performance such as the area under the ROC curve.22 We chose a lower limit of 30% to rule out asthma and an upper limit of 60% to rule it in, which implies an area of indecision between the probabilities of 30% and 60%. We think that these limits — or limits close to these — may be relevant in daily primary care practice.23 However, evidence-based thresholds would require formal benefit-harm or cost-utility analyses that incorporate treatment effects, potential harms (and costs), and children's (and their parents') valuation of all different health outcomes. Although utility measurement in children is not simple, instruments to measure quality of life in children do exist.24,25

Interpretation of findings in relation to previously published work

The API represents innovative research and is the most investigated index. This index was developed in the Tucson general population cohort9 and has also been tested in other populations (e.g. in the Leicester Respiratory Cohort, n=1,954). Unfortunately, one of the predictors — a blood sample for eosinophilia at one year of age — was omitted from the Leicester cohort.26

Several predictive indices for preschool children have been published in the past years.9,​10,​11,​12,​13,27,​28,​29,​30,​31 However, the populations in which these indices were developed are atypical of primary care, which may limit their application in the relevant spectrum of high-risk children in primary care. Nevertheless, most indices were prospectively developed and a number of selected predictors correspond reasonably well with ours: family history of atopy or asthma, persistent airway complaints (mainly wheezing), eczema, allergic rhinitis, and allergic blood testing. An important difference between the CAPS and other indices is the endpoint: only Eysink et al. developed their index with a comparable outcome — namely, a strict definition of the diagnosis of asthma.29 The major difference between the CAPS and the prediction index of Eysink et al. is that we used clinically more relevant inclusion criteria, which is reflected in the higher overall prevalence of asthma. The other published indices used varying ‘asthma’ definitions, mainly persistent wheezing at a specific age (varying from six to 13 years),9,​10,​11,​12,​13,28,31 or asthma defined by asthma symptoms or a physicians' diagnosis of asthma at age six to 10 years.9,27,30,31 Using weak and varying definitions may cause variation in prevalences and prediction indices, hampering generalization of findings.23

Most research groups used different populations to develop their indices and did not focus on the target population (i.e. preschool children presenting with respiratory complaints to their primary care physician). Eysink et al. included children aged one to four years visiting their primary care physician for persistent coughing of at least five days, and followed them to develop a predictive index for persistent asthma.29 Vial Dupuy et al. retrospectively selected wheezing infants from the medical files of an outpatient department of a paediatric pulmonology and allergy centre. The present study resembles the earlier Dutch study in the primary care setting except that the previous study was retrospective and the children were younger.31 Frank et al. selected their patients by postal survey on parent-reported wheeze, although this matched the clinicians' findings in fewer than 50% of cases.11,32 In the seven remaining studies, the participants were selected from general population birth cohorts,9,10,12,13,27,28,30 which makes these indices of doubtful applicability to most clinical settings unless major adaptations are made.

The predictive abilities of existing indices are, in general, not high. For example, Castro-Rodriguez et al. claim that the API is a ‘very useful tool’ to predict the long-term outcome of preschool wheeze ‘in every healthcare setting worldwide’. However, it was calculated that the performance of the API in more high-risk populations would be poorer.15 A negative API result in the Tucson cohort (‘asthma’ prevalence at age six of 22.2%) yielded a 10% probability of asthma (NPV=0.90). Overall, earlier predictive indices also had relatively high NPVs but disappointing PPVs, which makes them more useful to rule out asthma than to rule it in.26

Implications for future research, policy and practice

The implementation of a simple risk score in daily care is attractive to both physicians and parents. It allows physicians to provide tailored medical care and follow-up in children at high risk of developing asthma, and to apply a wait-and-see policy in those with a low or intermediate risk of asthma. The increased likelihood of predicting which children will develop asthma will help physicians to explain treatment and prognosis to parents, reducing uncertainties and improving adherence in their children.

Our study was a model development study, extended with internal validation using bootstrapped variable selection and shrinkage of regression coefficients, with the aim of developing a multivariable prediction model. Before the model can be confidently applied in daily clinical practice, external validation is needed.18 Also, further research should focus on new markers or tests to decrease the number of patients remaining inside the area of indecision (i.e. to increase discrimination).

Risk stratification serves three distinct functions in controlled trials: (1) assembling the appropriate spectrum of patients and allowing more efficient subgroup analysis14 and better assessment of baseline comparability; (2) it may also distinguish persistent and transient complaints early on and therefore increase the accuracy of the RCT results;3 and (3) the CAPS may be used to re-analyse early intervention trials to assess treatment response variability as a function of the CAPS score.3 Such trials should, of course, allow the calculation of a CAPS score.

Conclusions

We have developed an easy-to-use CAPS for preschool children at high risk of asthma in a primary healthcare setting. After suitable validation, the CAPS may assist in guiding shared decision-making to tailor the need for medical or non-medical interventions.

References

  1. 1.

    , , , , . Intermittent inhaled corticosteroids in infants with episodic wheezing. N Engl J Med 2006;354(19):1998–2005.

  2. 2.

    , , , et al. Long-term inhaled corticosteroids in preschool children at high risk for asthma. N Engl J Med 2006;354(19):1985–97.

  3. 3.

    , . Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA 2007;298(10):1209–12.

  4. 4.

    , , , et al. Prediction and treatment of asthma in preschool children at risk: study design and baseline data of a prospective cohort study in general practice (ARCADE). BMC Pulm Med 2009;9:13.

  5. 5.

    . Identifying and managing the infant and toddler at risk for asthma. J Allergy Clin Immunol 2010;126(2):417–22.

  6. 6.

    , . Additional patient outcomes and pathways in evaluations of testing. Med Decis Making 2009;29:E30–8.

  7. 7.

    , , , et al. Eight-year outcomes of the Childhood Asthma Prevention Study. J Allergy Clin Immunol 2010;126(2):388–9.

  8. 8.

    , , , et al. Associations and interactions of genetic polymorphisms in innate immunity genes with early viral infections and susceptibility to asthma and asthma-related phenotypes. J Allergy Clin Immunol 2012;130(6):1284–93.

  9. 9.

    , , , . A clinical index to define risk of asthma in young children with recurrent wheezing. Am J Respir Crit Care Med 2000;162(4 Pt 1):1403–06.

  10. 10.

    , , , et al. Predicting the long-term prognosis of children with symptoms suggestive of asthma at preschool age. J Allergy Clin Immunol 2009;124(5):903–10.

  11. 11.

    , , , , . Long term prognosis in preschool children with wheeze: longitudinal postal questionnaire study 1993-2004. BMJ 2008;336(7658):1423–6.

  12. 12.

    , , , . Predicting persistent disease among children who wheeze during early life. Eur Respir J 2003;22(5):767–71.

  13. 13.

    , , , et al. Wheezing in childhood: incidence, longitudinal patterns and factors predicting persistence. Eur Respir J 2008;32(3):585–92.

  14. 14.

    , , , . Predicting who will have asthma at school age among preschool children. J Allergy Clin Immunol 2012;130(2):325–31.

  15. 15.

    . The Asthma Predictive Index: not a useful tool in clinical practice. J Allergy Clin Immunol 2011;127(1):293–4.

  16. 16.

    , , . Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med 2004;23(10):1631–60.

  17. 17.

    , , . Multiple imputation using chained equations: issues and guidance for practice. Stat Med 2011;30(4):377–99.

  18. 18.

    , , , , . Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004;23(16):2567–86.

  19. 19.

    , , , . Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008;27(2):157–72.

  20. 20.

    British Thoracic Society and Scottish Intercollegiate Guidelines Network. British Guideline on the management of asthma. . Last updated Jan 2012 (accessed 4 Mar 4 2013).

  21. 21.

    , , , et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 2003;138(1):W1–12.

  22. 22.

    , , , , , . Evaluation of markers and risk prediction models: overview of relationships between NRI and decision-analytic measures. Med Decis Making 2013;33(4):490–501.

  23. 23.

    , , , , , . Different definitions in childhood asthma: how dependable is the dependent variable? Eur Respir J 2010;36(1):48–56.

  24. 24.

    . Challenges in health state valuation in paediatric economic evaluation: are QALYs contraindicated? Pharmacoeconomics 2011;29(8):641–52.

  25. 25.

    , , , , . Minimum skills required by children to complete health-related quality of life instruments for asthma: comparison of measurement properties. Eur Respir J 1997;10(10):2285–94.

  26. 26.

    , , , , , . Validation of the Asthma Predictive Index and comparison with simpler clinical prediction rules. J Allergy Clin Immunol 2011;127(6):1466–72.

  27. 27.

    , , , et al. Severity of obstructive airways disease by age 2 years predicts asthma at 10 years of age. Thorax 2008;63(1):8–13.

  28. 28.

    , , , et al. Toward improved prediction of risk for atopy and asthma among preschoolers: a prospective cohort study. J Allergy Clin Immunol 2010;125(3):653–9.

  29. 29.

    , , , et al. Accuracy of specific IgE in the prediction of asthma: development of a scoring formula for general practice. Br J Gen Pract 2005;55(511):125–31.

  30. 30.

    , , , et al. Asthma prediction in school children; the value of combined IgE-antibodies and obstructive airways disease severity score. Allergy 2010;65(9):1134–40.

  31. 31.

    , , , , . A simple tool to identify infants at high risk of mild to severe childhood asthma: the persistent asthma predictive score. J Asthma 2011;48(10):1015–21.

  32. 32.

    , , . What do parents of wheezy children understand by “wheeze”? Arch Dis Child 2000;82(4):327–32.

Download references

Acknowledgements

Handling editor Niels Chavannes

Statistical review Gopal Netuveli

We would like to thank the general practitioners at HAG-net-AMC, Zorggroep Almere and Prinsenhof, Leemhuis, and Buitenhuis for their time and effort. We also thank all parents and children for participating in the study. Our thanks also go to Pauline van Steenwijk, Machteld IJff, Alice Karsten, and Albertien Buijs for their invaluable help with data collection and other logistics.

Funding This study was financially supported by the Netherlands Asthma Foundation (3.4.06.078) and Stichting Astma Bestrijding (2008/027).

Author information

Affiliations

  1. Department of General Practice, Academic Medical Center, Amsterdam, The Netherlands

    • Lonneke B van der Mark
    • , Karina E van Wonderen
    • , Jacob Mohrs
    •  & Gerben ter Riet
  2. Department of Pediatric Respiratory Medicine and Allergy, Emma Children's Hospital — Academic Medical Center, Amsterdam, The Netherlands

    • Wim MC van Aalderen
  3. Department of General Practice, Erasmus Medical Center, Rotterdam, The Netherlands

    • Patrick JE Bindels

Authors

  1. Search for Lonneke B van der Mark in:

  2. Search for Karina E van Wonderen in:

  3. Search for Jacob Mohrs in:

  4. Search for Wim MC van Aalderen in:

  5. Search for Gerben ter Riet in:

  6. Search for Patrick JE Bindels in:

Contributions

LvdM was the lead investigator, participated in the design, coordinated the follow-up of the study, carried out all the statistical analyses, and drafted the manuscript. KEvW participated in the design, coordinated the follow-up of the study, and revised the manuscript critically. JM and WMCvA participated in preparing the data for statistical analysis and revised the manuscript critically. GtR participated in the design and coordination of the study, supervised it, participated in the statistical analysis, helped to draft the manuscript, and revised the manuscript critically. PJEB participated in the design of the study, supervised it, and revised the manuscript critically. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no conflicts of interest in relation to this article.

Corresponding author

Correspondence to Lonneke B van der Mark.

Appendices

Appendix 1. Supplementary material