Introduction

Since 2005, international asthma guidelines have recommended that asthma management should be guided by an assessment of asthma control,1,2 and asthma control is increasingly being used as an outcome measure in clinical trials. For this purpose, several composite asthma control tools, including the 5, 6 or 7-item Asthma Control Questionnaire (ACQ)3 and the 5-item Asthma Control Test (ACT),4 have been developed and validated in a range of populations against physician assessment of asthma control3,5–8 and guideline-based classifications.9,10 Both the ACQ and ACT have been recommended as core outcomes for clinical research by the American Thoracic Society/European Respiratory Society Task Force on Assessment of Asthma Control and Exacerbations11 and the NIH Asthma Outcomes Workshop.12

However, composite scoring systems are not necessarily suitable for primary care contexts such as community pharmacy or general practice where there are substantial pressures on clinicians' time. For such settings, there is considerable interest in brief screening questionnaires that can rapidly identify patients for whom more detailed assessment is needed. Existing screening tools include the Royal College of Physicians' ‘3 questions’ (RCP-3) developed in the UK13,14 and the 30-second Asthma Test developed in Canada.15

In a previous randomised controlled trial,16 a brief tool based on then asthma guidelines17 was used by community pharmacists to assess patients' asthma status. The opportunity to examine the performance of this tool came in a recent multicentre randomised controlled implementation trial of a comprehensive pharmacy-based asthma programme.18,19 In this study, asthma control was assessed at baseline and at the end of the study using both the ACQ-6 and the brief asthma control tool, hereafter called the Pharmacy Asthma Control Screening (PACS) tool.

The aim of the present analysis was to describe the accuracy of the PACS for identifying patients with suboptimal asthma control in patients considered to be at risk of adverse asthma outcomes compared with a validated asthma control tool, the 6-item ACQ (ACQ-6).

Methods

Data were obtained from a multicentre randomised controlled implementation trial which was conducted in urban and rural areas of four states/territories of Australia in 2009–10 to compare the effectiveness of a comprehensive asthma management service delivered by community pharmacists at three versus four visits over six months. Full details are reported elsewhere.18,19 The study protocol was approved by the Human Research Ethics Committees of the University of Sydney, Charles Sturt University, the University of Queensland and Monash University. All pharmacists and patients provided written informed consent.

Inclusion and exclusion criteria

Pharmacists recruited patients between January and May 2009. Patients were eligible to participate if they were aged ≥18 years, had a previous diagnosis of asthma, were able to attend the pharmacy for follow-up over six months, and were considered to be at risk of poor asthma outcomes by one or more of the following criteria:16 (a) using reliever more than three times per week in the previous four weeks (i.e. consistent with suboptimal asthma control);20 (b) fulfilling one or more criteria from a modification of the revised Jones Morbidity Index;21 or (c) not having seen their doctor for asthma in the previous six months.

Patients were excluded if they had a terminal illness, did not speak English well enough to communicate with the pharmacist or complete the study questionnaires independently, were enrolled in another study, or did not self-administer their asthma medications.

Patient assessments

Asthma control was assessed at the baseline visit before any education had been provided and at the final visit for the main study six months later. Two asthma control assessment tools were used. One was the PACS, similar to the tool used in the previous community pharmacy study.16 This was completed by the pharmacist based on five criteria for self-reported frequency of symptoms and activity limitation in the previous month (Tables 1 and 2), with categorisation of asthma control into good, partial, and poor.

Table 1 Original Pharmacy Asthma Control Screening tool (PACS)
Full size table
Table 2 Simplified PACS tool (suggested name for broader clinical use: “Primary care Asthma Control Screening tool”)
Full size table

Patients also completed on paper the validated ACQ-6.6 This comprises six questions, five about symptoms and one about frequency of daily reliever use. Text responses for each item are scored from 0 to 6, and the ACQ-6 score (0–6) is the average of the individual items with 0 representing no impact from asthma and 6 representing extremely poorly-controlled asthma.3 The crossover between ‘well-controlled’ and ‘not well-controlled’ asthma for ACQ-6 is reported to be close to 1.00,5,9 and a score of ≥1.50 indicates a high probability of poorly-controlled asthma.5,9 The minimal clinically important difference for ACQ-6 is 0.5.6

Spirometry was recorded by the pharmacists using EasyOneTM spirometers (Niche Medical, Sydney, Australia). All data were collected by pharmacists and stored in secure patient files. Data were de-identified, coded, and analysed by the researchers. ACQ-6 scores were calculated by the investigators.

Data analysis

Data analyses were performed using Analyse-It version 2.26 and SAS. Descriptive statistics were used to summarise patient characteristics, PACS category, and ACQ-6 data at baseline and six months. For all other analyses, PACS category was dichotomised to poor and good/partial. Sensitivity and specificity of the PACS ‘poor control’ category for identifying patients with suboptimally controlled asthma were calculated for ACQ 6 ≥1.0 and ≥1.5,5,9 with adjustment for clustering by pharmacy and calculation of 95% confidence limits.22 Agreement between PACS category and ACQ-6 category was calculated at six months using the κ test. Change in asthma control was analysed using the Wilcoxon signed rank test for ACQ-6 and the χ2 test for PACS. Agreement between the two tools for change in asthma control (change of ≥0.5 in ACQ-6,6 change in PACS category) was assessed using the κ test. The significance level was set at p<0.05.

Results

Ninety-six pharmacists recruited 570 people with asthma, 398 of whom (70%) completed the six-month pharmacy programme. No differences in any outcomes were seen between patients receiving the asthma programme at three pharmacy visits compared with four visits, so data from both randomisation groups were combined for the present analyses. Full details are reported elsewhere.19

Population characteristics

Table 3 shows the baseline characteristics of the enrolled study population (n=570). The average age was 51 years and 62% of participants were female. Mean forced expiratory volume in one second was 75.2±21.5% predicted. The majority of patients were using a combination inhaled corticosteroid/long-acting β2-agonist (ICS/LABA) at enrolment, consistent with national Australian data.23

Table 3 Baseline characteristics of the enrolled population (n=570)
Full size table

Assessment of asthma control at baseline and six months

At baseline the PACS category was available for 569 patients and ACQ-6 scores for 555 patients (Table 3). One ACQ score was out of range and was excluded. The average ACQ-6 score at baseline was 1.58±1.05 (n=554). Figure 1A shows the distribution of ACQ scores for the PACS categories at baseline; 77% of patients were classified by the PACS tool as having poor control. The sensitivity of PACS ‘poor control’ for identifying patients with ACQ-6 ≥1.0 (not well-controlled asthma) was 0.92 (95% confidence interval (95% CI) 0.89 to 0.95) and specificity was 0.66 (95% CI 0.58 to 0.74) (Table 4).

Figure 1
figure 1

Asthma Control Questionnaire-6 (ACQ-6) score distribution for Pharmacist Asthma Control Screening tool (PACS) categories at (A) baseline and (B) six months. Dotted lines show ACQ-6 score of 1.0 (crossover between well-controlled and not well-controlled asthma) and 1.5 (above which there is a high probability that asthma is poorly controlled). Numbers indicate median ACQ-6 values for each PACS category

Full size image
Table 4 Performance of the Pharmacy Asthma Control Screening tool (PACS) ‘poor control’ against Asthma Control Questionnaire (ACQ-6)
Full size table

At six months the average ACQ-6 score had improved to 0.96±0.88 (n=392). Figure 1B shows the distribution of ACQ scores for PACS categories at six months, with 40% classified by the PACS tool as having poor control. The sensitivity of PACS ‘poor control’ for identifying patients with ACQ-6 ≥1.0 was 0.76 (95% CI 0.68 to 0.83) and specificity was 0.83 (95% CI 0.77 to 0.89). Agreement between PACS ‘poor control’ and ACQ-6 ≥1.0 was moderate (κ=0.54).

The sensitivity and specificity of PACS for ACQ-6 ≥1.5 (high probability of poorly-controlled asthma) are also shown in Table 4.

Change in asthma control between baseline and six months

The median change in ACQ-6 between baseline and six months was 0.5 (Wilcoxon signed rank test p<0.0001), with 48% of patients having a clinically important decrease of 0.5 or more in ACQ-6 score. There was a significant change in PACS category (χ2 22.2, p<0.0001); of patients with PACS ‘poor control’ at baseline, 53% improved to good or partial control by the end of the study. Agreement between improvement in PACS category (dichotomised to poor and good/partial) and a decrease in ACQ-6 score of 0.5 or more was fair (κ=0.31), with 70/187 patients (37%) who achieved an improvement in ACQ-6 score of ≥0.5 being in the same (or worse) PACS category.

Discussion

Main findings

This study, based on a prospective analysis from a multicentre randomised controlled implementation trial in community pharmacy, showed that a simple PACS tool was feasible for the rapid assessment of asthma status in community pharmacies. When the performance of the PACS tool was evaluated against the ACQ-6, a validated tool primarily used in clinical research,6 PACS poor control had good sensitivity for identifying patients with not well-controlled asthma (ACQ-6 ≥1.0) — that is, 0.92 at baseline when most patients had poorly-controlled asthma and 0.76 after a six-month pharmacy asthma management programme when there was a broader distribution of asthma control. However, agreement between change in the two assessments was only fair, and many patients who were still classified by PACS as having poor control at the end of the study nevertheless had achieved a clinically important improvement in ACQ-6 score since enrolment.

Interpretation of findings in relation to previously published work

The PACS tool comprises five items covering four areas: asthma symptom frequency, night waking, chest tightness on waking, and activity limitation. Similar domains are included in current guideline-based assessment of asthma2,20 and in many asthma control assessment tools, although asthma symptoms during the night and upon waking are often combined. Factor analysis has shown that reliever frequency associates strongly with symptom frequency,24 and the ACQ itself performs similarly with and without the reliever question.5,6 The present results showed that, in the end-of-study population which had a broad range of ACQ-6 scores, the sensitivity of PACS poor control for not well-controlled asthma (ACQ-6 ≥1.0) or for poorly-controlled asthma (ACQ-6 ≥1.5) was moderately high, as is desirable for a screening tool; specificity was also high, indicating that few patients would be inappropriately identified with the use of the screening tool. This level of agreement was particularly good given that PACS questions are about symptoms in the previous month and responses are based on symptom frequency whereas ACQ questions are about symptoms in the previous week and most items are scored more subjectively on the intensity or magnitude of symptoms.

The main difference between the PACS criteria and current guidelines-based assessment lies in the frequency cut-off points for asthma symptoms over the past four weeks, with guidelines20,25 permitting symptoms twice a week in well-controlled asthma compared with none at all for PACS-based ‘good’ control and once a week for ‘partial’ control. Although in the past there has been an emphasis on ‘total’ control, there is increasing recognition that this cannot be achieved by around 45% of patients even with high-dose ICS/LABA therapy under clinical trial conditions,26,27 and that medication doses should not be pushed relentlessly higher if there are occasional residual symptoms.28 Based on guidelines criteria, we made an a priori decision to group ‘good’ and ‘partial’ control together for all but the descriptive analyses, and we recommend that this is the way the PACS tool should be used.

The relationship between change in ACQ and change in asthma control category has been examined by O'Byrne and colleagues using the Global Initiative for Asthma (GINA) control classification.9 In that analysis, as in the present study, many patients whose asthma was still classified as poorly controlled at the end of the study nevertheless achieved a clinically important improvement in ACQ score. These two studies indicate that a continuous scale such as the ACQ may be more responsive to change in asthma control than a categorical control classification, and that a continuous measure would be more appropriate for clinical research or clinical trial settings if a primary goal was detection of change.

For rapid identification in primary care of patients with asthma control problems, other examples of simple screening tools include the RCP-3,13 the Asthma Therapy Assessment Questionnaire (ATAQ),29 and the 30 second Asthma Test.15 The RCP-3 is the shortest, and asks about any difficulty sleeping due to asthma, daytime asthma symptoms, and activity limitation over either one week or one month. The performance of the RCP-3 was assessed by Pinnock and colleagues in a cross-sectional study in general practice. Of 129 patients with both ACQ-6 and RCP-3, 43% had ACQ-6 ≥1.00, a similar prevalence to the six-month data in the present study. The sensitivity and specificity of RCP-3 for ACQ ≥1.00 varied with the number of positive questionnaire items.14 An RCP-3 score of ≥1 had a sensitivity of 0.96 and specificity of only 0.34 for ACQ-6 ≥1.0 compared with 0.76 and 0.83 for PACS poor control in the present study. Similar patterns were seen between RCP-3 and ACQ-6 ≥1.5. The 30-second Asthma Test has three criteria for poor control assessed over one week and two items over three months. In a hospital clinic-based study (n=81), this tool demonstrated moderate correlation with specialist assessment of asthma control.15 The ATAQ, which is copyrighted to Merck & Co, has four items assessed over four weeks; in a health maintenance organisation study (n=4,788) it was shown to be strongly predictive of future healthcare utilisation.29

Responsiveness data do not appear to have been published for any of these other screening tools.

Strengths and limitations of this study

The strengths of the present study lie in the use of data from a large pragmatic multicentre implementation study in community pharmacies across a wide area of urban and rural Australia, the direct comparison with a validated asthma control tool, the independence of collection of the two tools (pharmacists asked the PACS questions and recorded each patient's PACS category, but patients self-completed the ACQ and the ACQ-6 score was calculated by study staff), and the examination of both cross-sectional and longitudinal performance. The limitations of the study are that most participants were selected for being at risk of adverse asthma outcomes rather than being from a general asthma population, and no independent physician assessment of asthma control was available.

The strengths of the PACS tool are its brevity, making it feasible for use in a busy clinical environment, the standardised timeframe (one month), and the good sensitivity and specificity for identifying patients whose asthma is not well-controlled. In particular, the performance of PACS with regard to specificity was substantially better than has been reported for RCP-3, avoiding over-interpretation of infrequent symptoms. Its limitations are that the PACS criteria are more stringent than the control criteria in current asthma guidelines, so PACS ‘poor control’ will identify some patients who, on more detailed assessment such as with the ACQ-6, will be found to have well-controlled asthma. In addition, the inclusion in the PACS tool of two grades of activity limitation may partly explain the poor agreement for change in asthma control, since an improvement from limitation on moderate exercise to limitation only on vigorous exercise would not change the patient's PACS control classification.

Implications for future research, policy and practice

Future clinical research with PACS could be carried out to confirm the effect on its performance of removing the vigorous activity item and of harmonising the symptom frequency criterion for good control (no symptoms in four weeks) with that accepted in current guidelines (twice/week or less).

Based on current guidelines criteria for well-controlled asthma and the sensitivity and specificity findings from the present analysis, we recommend that the PACS ‘good’ and ‘partial’ control categories should be combined into a single category of ‘well-controlled asthma’. When this change is operationalised (i.e. with each PACS item dichotomised around a frequency of more than once a week), the PACS tool becomes even simpler for administration, with five short yes/no questions as shown in Table 2. There is no reason why the use of the PACS should be limited to pharmacy practice, so it may be more appropriate to refer to it as the Primary care Asthma Control Screening tool.

For clinical practice, the present findings show that the PACS tool is feasible for rapid screening in primary care, and that it can identify a high proportion of patients who have poorly-controlled asthma by standardised ACQ-6 criteria. However, health professionals using PACS (or other screening tools) need to be made aware that, in order to maximise sensitivity, the control criteria are more stringent than in current guidelines. Hence ‘poor control’ by the PACS tool should prompt further asthma assessment rather than necessarily indicating that a change in treatment is required.

For monitoring of change in asthma control, PACS-based control assessment was responsive to a pharmacy-based asthma management program in both this and our previous study.16 However, it was significantly less responsive than the continuous ACQ-6 score, as is also the case for other categorical control classifications such as GINA.9 This suggests that the primary use of PACS should be as a screening tool rather than for long-term monitoring. For regular monitoring of asthma control in clinical practice, one option would be to record three simple numerical measures for symptom frequency (days/week), night waking (nights/week) and activity limitation (number of flights of stairs or city blocks able to be walked); another option would be to use a numerical asthma control tool such as the ACQ.

Finally, as with any other asthma control instrument, it is important that the PACS tool should not be used in isolation, including for patients classified as having ‘poor control’. A more detailed assessment of asthma control and other clinical factors (including co-morbidities, risk factors for exacerbations, and side-effects) and patient preference should also be taken into account in considering what further investigations and management may be needed.

Conclusions

The PACS is a simple screening tool which is feasible for brief screening of asthma status in primary care, such as in community pharmacy. It has good sensitivity and specificity for identifying patients whose asthma is not well-controlled, in whom more detailed assessment of asthma status is indicated.