Introduction

Childhood asthma usually starts during the preschool years. Unfortunately, it is currently not possible to test for or alter the natural history of asthma accurately in young children,1,2 so asthma during the preschool years is frequently underdiagnosed. About 10 asthma risk prediction indices for young children have been developed, although only one was developed in a primary care setting where most of these children present with respiratory symptoms for the first time and where most are treated. Risk stratification using prediction indices may be useful for clinical and research purposes. For example, risk stratification supports the tailoring of treatment strategies such that benefits and harms may be properly weighed.3 It may therefore improve the follow-up of children at risk by decreasing the disease burden and improving management, resulting in fewer respiratory symptoms, exacerbations, and emergency medical visits.4,5

Prediction indices are valuable because the information itself may contribute to patients' quality of life.6 Patients' emotional response to medical testing is a natural effect. The information involved in a high predicted probability of a young child having asthma at six years of age may enable them (or their parent/guardians) to make sense of the symptoms or to adjust their lifestyle and living conditions in light of the results, while a low predicted probability may alleviate their concerns.6,7

In addition, various research groups are investigating the underlying pathogenic mechanisms and preventive strategies for asthma,8 and as soon as effective therapy that is either costly and/or has adverse effects emerges, risk stratification will be useful.3

Existing predictive indices have an uncertain value for the primary healthcare setting because the majority of the cohort studies were based on parental questionnaires in a general population,9–13 so they are of modest clinical value for preschool children.14 For example, the most well-known indices — the loose and stringent Asthma Predictive Indices (API) — were developed in a general population and are based on questionnaires.9 API predicts persistent wheezing in children who ever wheezed and does not predict asthma in children with recurrent wheeze presenting to a physician.15

In 2004 we started a prospective cohort of young children at high risk of developing asthma. The objective was to construct a clinical index for predicting asthma in preschool children at risk of asthma presenting in primary care. Here we describe the development and performance of a novel clinical asthma prediction score (CAPS).

Methods

AppendixDesign and subjects

The subjects were children aged one to five years in the Netherlands who were participating in the ARCADE (AiRway Complaints and Asthma Development) prospective cohort study.4 They had presented at one of the pertinent 14 primary care clinics in the previous 12 months with complaints of recurrent coughing (≥2 visits), wheezing (≥1 visit), and/or shortness of breath (≥1 visit). The study was approved by the Netherlands Central Committee on Research Involving Human Subjects (CCMO/P04.0098C) and the details of the study have been published elsewhere.4 Written informed consent was obtained from the parents before all measurements.

Data collection

Questionnaires

For this study we derived demographic data and data with respect to the child's clinical history from questionnaires obtained by the Core Questionnaire of the International Study of Asthma and Allergy in Children (ISAAC), which was administered at baseline and at the age of six years.

Allergy test

Total immunoglobulin E (IgE) and specific IgE directed against cat, dog, and house dust mite were determined by radioallergosorbent test (RAST) in all the children at baseline. IgE positivity to cat, dog, and house dust mite was defined as >0.35kU/L.4

Spirometry and bronchial hyper-responsiveness

At six years of age, spirometry and bronchial hyper-responsiveness (BHR) assessments were obtained in children with wheezing, shortness of breath, recurrent coughing, or use of asthma medication (β2-agonists, inhaled corticosteroids) during the previous 12 months. BHR to increasing doses of methacholine was measured according to Cockcroft et al.(online supplement reference 3).

Outcome: asthma diagnosis at age six

The diagnosis of asthma at six years of age was defined as: having persistent symptoms of asthma and/or using asthma medication in the last year in combination with BHR (defined as the methacholine dose (<8.0mg/ml) causing a 20% reduction in forced expiratory volume in one second (PC20-FEV1)) or reversibility (>10% increase in FEV1 after short-acting β2-agonists).4 Persistent symptoms were defined as self-reported complaints during the previous 12 months of wheezing and/or shortness of breath and/or recurrent coughing.

Children who had not experienced asthma-related symptoms in the previous year or who had not used asthma medication in the previous year were not invited for lung function measurement. They were considered as not having asthma.

Probability of asthma at the age of six

Depending on the direction and magnitude of the selected predictors as measured by regression coefficients, we developed the Clinical Asthma Prediction Score (CAPS) as a points system which is easy to use, without requiring a computer or calculator.16 To use the CAPS in clinical practice, thresholds are needed to either rule in or rule out asthma at the age of six years. Therefore we set two thresholds — a lower limit at 30% below which we assumed that many physicians may well choose a ‘wait and see’ policy, and an upper limit at 60% above which it was assumed that physicians may pursue a more active management such as a trial of asthma treatment. These thresholds imply that children with a predicted risk of asthma risk between 30% and 60% will be assigned to the area of clinical indecision in which a ‘watchful waiting’ policy may be defensible.

Statistical analysis

Details of the statistical analysis are described in Appendix 1, available online at www.thepcrj.org. Briefly, we analysed whether dropout was selective and missing predictor data were imputed using multiple imputation.17 Twelve candidate predictors of all available data were preselected based on clinical expert opinion after removing five due to collinearity (Table 1). Bootstrapped (1000x) backward selection was used to select the final important predictors whose regression coefficients were shrunk to facilitate better performance when the model is applied to other data or future patients.18 The predictive performance of the prediction index was assessed by its calibration and discrimination abilities.

Table 1 Candidate predictors of asthma diagnosed at 6 years of age in preschool children
Full size table

Furthermore, the additional value of specific IgE was calculated by the additional area under the ROC curve (AUCdiff) and the net reclassification improvement (NRI).19 All statistical analyses were carried out in Stata/SE Version 10.1 (Stata Corp, College Station, TX, USA).

Results

Of the 771 children, 438 (56.8%) had complete information at baseline and an asthma diagnosis at the endpoint (Figure 1). Of these, 187 (42.7%) had asthma at the age of six. Table 1 shows the univariable frequencies of the preselected candidate predictors. The mean (SD) age at baseline was 3.00 (1.29) years and 189 (43.1%) were female. In 156 children (35.6%), asthma was present in either the parents or siblings. Missingness analyses showed no strong indications for selective drop out (Appendix 1 available online at www.pcrj.org).

Figure 1
figure 1

Flow diagram

Full size image

Multivariable analysis

Five predictors were included (Table 2): age, family history of asthma or allergy, wheezing-induced sleep disturbances in the previous 12 months, wheezing in the absence of a cold in the previous 12 months, and specific IgE test. The discrimination of the model expressed as area under the ROC curve was 0.73 (SE=0.024). Calibration was satisfactory (Table 2).

Table 2 Predictors for asthma at age of six years and prediction model performance measures
Full size table

IgE and no IgE in the model

The model performance was calculated for the model including age, family history, wheezing-induced sleep disturbances, and wheezing in the absence of a cold but excluding IgE. The discrimination of the model expressed as area under the ROC curve was 0.71 (SE=0.024). Calibration was good (Table 2). Adding specific IgE to the model increased the AUC by 2.8 percentage points (AUCdiff= 0.028, 95% CI 0.0047 to 0.051, SE=0.012). Net reclassification analysis showed that 11.23% (p=0.09) of the children were correctly reclassified when IgE was added to the model based on history items only (Table 3).19

Table 3 Reclassification tables for model with or without IgE in children with and without asthma
Full size table

Probability of asthma at the age of six using CAPS

A points system ranging from 0 to 11 was developed. Thresholds to accept or reject the presence or development of asthma were set at 30% and 60%, respectively (Table 4).16 The positive predictive values (PPVs) of the CAPS (≥7=probability of asthma of ≥60%) were 74.3% (CAPS=7) and 100% (CAPS=11) and the negative predictive values (NPVs) of the CAPS (<3=probability of asthma of ≤30%) were 78.4% (CAPS=2) and 88.2% (CAPS=0) (see online supplement). Figure 2 shows asthma risks as a function of CAPS scores.

Table 4 Risk estimates associated with Clinical Asthma Prediction Score (CAPS) points
Full size table
Figure 2
figure 2

Probability of having asthma at age 6 years, including 95% confidence internvals, associated with CAPS

Full size image

Discussion

Main findings

The clinical asthma prediction score (CAPS) we developed in a prospective cohort study facilitates the risk stratification of preschool children who attend primary healthcare with wheezing, dyspnoea, or recurrent coughing. The predictive ability of the CAPS may assist physicians to distinguish between a relatively low risk (14%) and a very high risk (90%) of developing asthma. The CAPS uses easily accessible information on age, family history of asthma or allergy, wheezing-induced sleep disturbances, and wheezing in the absence of a cold, and an additional blood sample to obtain specific IgE.

The use of the CAPS may enable physicians to reduce the number of children unnecessarily labelled with asthma (low CAPS). For this group, the prior probability went from 42.7% to 21.6%; this helps to reduce unnecessary testing, treatment, and uncertainty. On the other hand, CAPS may allow physicians to identify preschool children who may need proper asthma management (high CAPS). For this group, the prior probability went from 42.7% to 74.3%; a high CAPS may therefore reduce the burden of asthma in these children through better symptom control and fewer exacerbations and hospital admissions. In both situations there will be a reduction in costs. Children with an intermediate CAPS (3>7) retain a probability similar to the probability before applying the CAPS. In this group a watchful waiting policy seems justified, in line with the British guideline,20 which subsequently suggests a watchful waiting approach with review in the majority of cases with intermediate risk predictions.

Strengths and limitations of this study

Our study has the following strengths. First, we selected children from a relevant population — namely, those children in whom an attending primary care physician had considered asthma. Second, all predictors collected are easily accessible in primary care. Third, we used a stringent definition of asthma to ensure a reliable reference standard diagnosis for research purposes. Fourth, we followed the relevant STARD checklist criteria — a strict protocol with well-defined inclusion and exclusion criteria for the setting and population and a generally accepted reference standard.21 Fifth, shrinkage of the magnitude of the regression coefficients was used to counteract over-optimism of our model, thus enhancing its validity outside our data set.17,18 Sixth, we used NRI to measure the additional value of specific IgE to the model. NRI is a marker to express the model performance and to compare models, focusing on the correct reclassification of patients with and without asthma; it appears to be a clinically more sensible approach than the differences between AUCs.

We see the following limitations to our work. First, our study is a model development study without external validation. We did, however, use bootstrapped variable selection and shrinkage of regression coefficients to temper an over-optimistic performance of CAPS in external populations. Second, we have a high dropout rate. Missingness analyses showed that dropout was higher in children of parents who smoked (OR=1.88) and those with persistent (>1 month) coughing (OR=1.62). Parental smoking may be explained as a psychological cause for dropout; parents who smoke may feel guilty or embarrassed that smoking might have a negative influence on their child's physical condition. We assumed that persistent coughing as a cause of dropout was a coincidental finding. The high dropout rate may also be explained by the fact that, due to an extension of follow-up, participants had to explicitly renew their informed consent. Third, although some combinations of predictor values yield low (<30%) or high (>60%) asthma probabilities, almost 58% of the children in our study population had a predicted probability between 30% and 60%, close to the overall prevalence of 42%. For such children, CAPS results are not likely to change clinical management. Finally, the choice of different thresholds may lead to different results. As far as we are aware, there is no evidence-based advice on thresholds for this type of decision-making in the asthma literature. We decided to use thresholds as a clinically intuitive alternative to more formal methods of reclassification and as an attractive alternative for classical methods of predictive performance such as the area under the ROC curve.22 We chose a lower limit of 30% to rule out asthma and an upper limit of 60% to rule it in, which implies an area of indecision between the probabilities of 30% and 60%. We think that these limits — or limits close to these — may be relevant in daily primary care practice.23 However, evidence-based thresholds would require formal benefit-harm or cost-utility analyses that incorporate treatment effects, potential harms (and costs), and children's (and their parents') valuation of all different health outcomes. Although utility measurement in children is not simple, instruments to measure quality of life in children do exist.24,25

Interpretation of findings in relation to previously published work

The API represents innovative research and is the most investigated index. This index was developed in the Tucson general population cohort9 and has also been tested in other populations (e.g. in the Leicester Respiratory Cohort, n=1,954). Unfortunately, one of the predictors — a blood sample for eosinophilia at one year of age — was omitted from the Leicester cohort.26

Several predictive indices for preschool children have been published in the past years.9–13,27–31 However, the populations in which these indices were developed are atypical of primary care, which may limit their application in the relevant spectrum of high-risk children in primary care. Nevertheless, most indices were prospectively developed and a number of selected predictors correspond reasonably well with ours: family history of atopy or asthma, persistent airway complaints (mainly wheezing), eczema, allergic rhinitis, and allergic blood testing. An important difference between the CAPS and other indices is the endpoint: only Eysink et al. developed their index with a comparable outcome — namely, a strict definition of the diagnosis of asthma.29 The major difference between the CAPS and the prediction index of Eysink et al. is that we used clinically more relevant inclusion criteria, which is reflected in the higher overall prevalence of asthma. The other published indices used varying ‘asthma’ definitions, mainly persistent wheezing at a specific age (varying from six to 13 years),9–13,28,31 or asthma defined by asthma symptoms or a physicians' diagnosis of asthma at age six to 10 years.9,27,30,31 Using weak and varying definitions may cause variation in prevalences and prediction indices, hampering generalization of findings.23

Most research groups used different populations to develop their indices and did not focus on the target population (i.e. preschool children presenting with respiratory complaints to their primary care physician). Eysink et al. included children aged one to four years visiting their primary care physician for persistent coughing of at least five days, and followed them to develop a predictive index for persistent asthma.29 Vial Dupuy et al. retrospectively selected wheezing infants from the medical files of an outpatient department of a paediatric pulmonology and allergy centre. The present study resembles the earlier Dutch study in the primary care setting except that the previous study was retrospective and the children were younger.31 Frank et al. selected their patients by postal survey on parent-reported wheeze, although this matched the clinicians' findings in fewer than 50% of cases.11,32 In the seven remaining studies, the participants were selected from general population birth cohorts,9,10,12,13,27,28,30 which makes these indices of doubtful applicability to most clinical settings unless major adaptations are made.

The predictive abilities of existing indices are, in general, not high. For example, Castro-Rodriguez et al. claim that the API is a ‘very useful tool’ to predict the long-term outcome of preschool wheeze ‘in every healthcare setting worldwide’. However, it was calculated that the performance of the API in more high-risk populations would be poorer.15 A negative API result in the Tucson cohort (‘asthma’ prevalence at age six of 22.2%) yielded a 10% probability of asthma (NPV=0.90). Overall, earlier predictive indices also had relatively high NPVs but disappointing PPVs, which makes them more useful to rule out asthma than to rule it in.26

Implications for future research, policy and practice

The implementation of a simple risk score in daily care is attractive to both physicians and parents. It allows physicians to provide tailored medical care and follow-up in children at high risk of developing asthma, and to apply a wait-and-see policy in those with a low or intermediate risk of asthma. The increased likelihood of predicting which children will develop asthma will help physicians to explain treatment and prognosis to parents, reducing uncertainties and improving adherence in their children.

Our study was a model development study, extended with internal validation using bootstrapped variable selection and shrinkage of regression coefficients, with the aim of developing a multivariable prediction model. Before the model can be confidently applied in daily clinical practice, external validation is needed.18 Also, further research should focus on new markers or tests to decrease the number of patients remaining inside the area of indecision (i.e. to increase discrimination).

Risk stratification serves three distinct functions in controlled trials: (1) assembling the appropriate spectrum of patients and allowing more efficient subgroup analysis14 and better assessment of baseline comparability; (2) it may also distinguish persistent and transient complaints early on and therefore increase the accuracy of the RCT results;3 and (3) the CAPS may be used to re-analyse early intervention trials to assess treatment response variability as a function of the CAPS score.3 Such trials should, of course, allow the calculation of a CAPS score.

Conclusions

We have developed an easy-to-use CAPS for preschool children at high risk of asthma in a primary healthcare setting. After suitable validation, the CAPS may assist in guiding shared decision-making to tailor the need for medical or non-medical interventions.