Main

Despite interventions being available, it is estimated that pneumonia is responsible for 15% of childhood deaths worldwide1. Reductions in annual mortality remain modest, with nearly 950,000 under-5 year olds dying of pneumonia in 2013 (ref. 2). Despite the unprecedented rate of Haemophilus influenzae type B (Hib) and pneumococcal vaccine (PCV) introduction, achieving high levels of coverage in developing countries is still challenging3. Therefore, in regions where vaccine introduction and scale-up lags behind other countries, improved access to diagnosis and treatment is crucial. This includes interventions at multiple points in the continuum of care — improving care-seeking practices, increasing the availability of suitable diagnostics, and guiding both formal and informal care providers in appropriate disease management. Unfortunately, current treatment coverage remains low, and, more importantly, most childhood pneumonia deaths result from a lack of, or delay in, accurate diagnosis4.

A crucial component of improving pneumonia outcomes is the early identification of patients at risk of treatment failure and the timely provision of supportive care. However, in the absence of appropriate prognostic tools at the frontline, currently recommended World Health Organization (WHO) guidelines for integrated management of childhood illness (IMCI) often lead to an overuse of antibiotics and the under-referral of patients with severe pneumonia who require hospital care5. The most recent 2015 technical update of IMCI guidelines defines non-severe pneumonia as the presence of fast breathing or chest in-drawing or both, which is treatable with oral antibiotics. Severe pneumonia is defined as cough or difficulty breathing in the presence of danger signs, and requires referral to a hospital or health facility for injectable antibiotics or other supportive care such as oxygen therapy6. Currently, identification of these IMCI symptoms remains inconsistent and unreliable among community health-care workers or carers without clinical training7. Therefore, improved prognostic and diagnostic tools for case-management are necessary to substantially reduce pneumonia-associated morbidity and mortality.

Hypoxaemia and malnutrition are strong predictors of mortality in children who are hospitalized for pneumonia8,9. This has led to increasing support for the use of oxygen therapy and monitoring oxygen saturation in the management of severe cases. It is estimated that 15% of children who are hospitalized for pneumonia have hypoxaemia (oxygen saturation, or SpO2, of <90% (ref. 10) and that around 1.5 million children with severe pneumonia require oxygen treatment each year11. The use of pulse-oximetry devices (used to measure the oxygen level in the blood) in community health-care settings has been proposed as a method to identify hypoxic children at risk of treatment failure. These devices may be particularly beneficial at the frontline given that they require little training and reduce the reliance on clinical symptoms. The current pulse-oximetry systems are also quick, non-invasive and require minimal infrastructure.

The aim of this study was to evaluate the public health impact and cost-effectiveness of current IMCI guidelines combined with pulse-oximetry devices as a prognostic tool in the hands of frontline health workers in resource-poor settings. To do this, we developed a model of disease progression that explicitly tracks the continuum of care pathways, and parameterized the model for the top 15 countries with the highest burden of pneumonia.

Methods

Model structure. The progression and treatment of pneumonia in a population of children under the age of five was modelled using a continuous-time deterministic compartmental model (Fig. 1). Full mathematical details of the model are given in the Supplementary Information. Without treatment, children may be in one of four states: susceptible (S), non-severe (NSV), severe (SV) or death (D). These definitions of non-severe and severe disease are different to those defined by IMCI. We classified severe disease as disease that requires hospitalization and non-severe disease as disease that can be successfully managed with oral antibiotics alone. Susceptible children become infected at a constant rate determined by the incidence of pneumonia in the population, and although most enter the non-severe state before progressing, a small proportion progress directly to severe disease. Children with non-severe disease may recover naturally (without treatment), progress to a treatment state or progress to the severe state. Children with severe disease may recover naturally, progress to a treatment state, or die and be tracked in the death state. Each of the non-severe disease and severe disease states are further subdivided into day of infection. Children with non-severe pneumonia who have not progressed to severe disease or recovered naturally after 14 days return to the susceptible state, whereas children with severe pneumonia who have not received treatment or recovered naturally after 14 days (giving a maximum potential illness length of 28 days) are assumed to have died.

Figure 1: Overview of model structure.
figure 1

The main states and transitions of the model are shown, with cases transitioning at time-dependent rates. Each state contains a subset of states to track the length of infection.

Full size image

As well as making a natural recovery, non-severe and severe cases may separately enter one of two treatment states: treatment received outside of hospital (referred to as community-based treatment) or hospital-based treatment, at rates that depend on the care-seeking rate and factors in the care pathway. The latter is determined by a decision tree in which four factors are included: availability of a prognostic tool, whether or not the prognostic tool gives the correct result, adherence to the prognosis (whether or not the correct treatment, according to the prognosis, was administered by the medical practitioner and followed by the patient) and treatment availability. An example of a section of the decision tree is given in Supplementary Figure 1. We aimed to capture correct and incorrect treatment rates of both non-severe and severe cases, so cases may move into either treatment state depending on the outcome of the decision tree, regardless of whether or not it is the correct treatment. Moreover, this design includes the possibility that cases can move into the correct treatment state even if the prognosis was incorrect.

Last, if treatment fails to work, then the case may move into one of two treatment failure states, determined by a probability of treatment success that depends on both the severity of the disease and whether the treatment is appropriate. For example, severe cases are less likely to be cured by community-based treatment than hospital-based treatment. Children with non-severe disease that fail to respond to treatment may progress to hospital treatment, to a severe state or naturally recover to the susceptible state. Children with severe disease who fail to respond to treatment may progress to hospital treatment, to the death state or naturally recover to the susceptible state at different rates to the non-severe cases.

Community-based treatment is assumed to consist of a course of amoxicillin that lasts for 3 days, whereas hospital-based treatment lasts for 7 days. To ensure that treatment does not prolong illness in the model, those who enter the treatment states at a late stage of infection (such as day 13 or 14 of non-severe illness) are assumed to recover before reaching the end of the treatment, instead of progressing through all 3 or more days of treatment and therefore taking longer to return to susceptible than if they are left untreated (see Supplementary Fig. 2).

Model parameters. The public health impact of the introduction of pulse-oximetry devices was evaluated at the community level in comparison to a baseline standard of care with IMCI by calculating the incremental deaths averted with the introduction of pulse oximetry in different countries. Countries included in the analysis were selected according to the number of pneumonia-attributed deaths that occur each year in that country, using estimates from 2011 (ref. 12). Although more recent estimates of pneumonia mortality in under-fives are available, these did not include estimates of the number of cases and so the 2011 estimates were used for the analysis. The 15 countries with the highest number of deaths were chosen, excluding China, Angola and Tanzania owing to a lack of data on the availability of community-based care in these countries.

The incidence of pneumonia in the model was fitted to the mortality data by finding the incidence rate that best matches the mortality data using a normal likelihood. The rate of progression from non-severe to severe disease was fitted to estimates of proportions of cases that progress to severe disease12,13. All other disease progression parameters were based on a review of the literature and expert opinion (Table 1). The availability of community-based care was fitted simultaneously to data on the percentage of children with suspected pneumonia receiving antibiotics3, assuming a binomial likelihood, whereas the other care-seeking parameters were obtained from the literature where available (Table 1). One care-seeking parameter — the probability of treatment with hospital care curing the case — could not be identified in the literature. For the purpose of assessing the impact of the pulse oximeter as a prognostic tool, we assumed that this was high, representing a situation in which oxygen and other facilities are available. Lower values of this parameter will reduce the impact and cost-effectiveness of any prognostic tool; this was explored in our sensitivity analysis.

Table 1 Model parameters. The central estimates shown are derived from the source literature with ranges added for the sensitivity analysis.
Full size table

To estimate the impact and cost-effectiveness of a new prognostic combination (IMCI and pulse oximetry), we also needed to make a number of assumptions about its availability, its ability to accurately classify a case as severe (sensitivity) or non-severe (specificity) compared with IMCI alone, and adherence to its use. For the purposes of assessing its utility, we assumed it would be made available and hence set this parameter to a high value. Although data were available to support the sensitivity of IMCI, sufficient data were not available to inform the sensitivity of IMCI when combined with pulse oximetry. Thus, we proposed two scenarios: one in which the addition of pulse oximetry increases the sensitivity of IMCI to 70% (referred to as PO1), and one in which the sensitivity of the combination is increased to 85% (referred to as PO2), reflecting the potential of pulse oximetry to identify both people with hypoxic cases and cases with abnormal oxygen saturation (90–95%) who would benefit from referral14. We could not find data on specificity and assumed these would be relatively high (85%) for both IMCI alone and the PO1 and PO2 prognostic packages (Table 1). Adherence to IMCI guidelines for both non-severe and severe cases was based on the literature, and we set severe prognosis adherence to the PO1 and PO2 prognostic packages to be higher (increased from 65% to 85%), reflecting the perception that a physical tool would increase the likelihood of adherence. Finally, two parameters are included to account for non-adherence to the prognosis. For those whose prognosis is severe disease, if the mother did not take the child to hospital despite referral we assumed that as a minimum the child would receive amoxicillin (taking into account its availability). For those with a non-severe prognosis, but for whom the treatment regimen was not adhered to, we assumed that they had a high probability of receiving no treatment.

Costing approach. The incremental cost-effectiveness of pulse oximetry was evaluated in comparison to a baseline of using IMCI alone. Costing was undertaken from a public health provider perspective and hence no societal, economic or private sector costs were included.

Costs were subdivided into two categories. Additional direct costs included the cost of prognostic equipment, batteries and delivery and distribution of the prognostic tool. Further health-care costs included additional amoxicillin courses for patients with non-severe disease, increased hospitalization costs for patients with severe disease and the cost savings that would arise from the reduction in inappropriate amoxicillin prescriptions or treatment (owing to assumed increased prognostic adherence) or hospital referral. Total costs are the direct costs plus the additional health-care costs. Wherever possible, costing data were obtained from the literature (Table 2). Health-care costs were obtained at the country level from the WHO–CHOICE database15. This included the average cost of an outpatient visit and the cost of inpatient stays based on an average of 7 days of hospitalization. The cost implications arising from potential overuse of amoxicillin (antibiotic resistance) were not included in the analysis given the difficulty of obtaining any meaningful quantitative estimate of this16. Disability-adjusted life years (DALYs) are calculated using country-specific life-expectancy values and are not discounted or age-weighted, as suggested by recent recommendations17. The overall cost-effectiveness compared with IMCI is then presented as the cost per DALY averted compared with IMCI.

Table 2 Cost parameters. The central estimates are shown derived from the source literature with ranges added for the sensitivity analysis.
Full size table

Sensitivity analysis. To explore sensitivity to key parameters, Latin hypercube sampling was used to draw 1,000 different parameter values from a triangular distribution. Outputs from the model were calculated using each set of parameters and the top and bottom 5% of the results were discarded to obtain 90% ranges. Additional sensitivity analyses are reported in the Supplementary Information.

Results

Figure 2a shows previous annual pneumonia mortality estimates12 and our estimates of the incidence of pneumonia per 1,000 children in the 15 countries with the highest burden. As expected, the estimated pneumonia incidence follows a similar pattern to mortality, but with variations owing to between-country variation in the availability of community-based care. Niger, Ethiopia and India are estimated to have particularly low rates of antibiotic treatment18, and thus are predicted by the model to have the highest case-fatality rates.

Figure 2: Pneumonia incidence and mortality.
figure 2

a, Estimated under-5 pneumonia incidence predicted by the model and previously reported mortality12 in top 15 countries with the highest burden. b, Median estimates of pneumonia deaths per 1,000 children under 5 across the 15 countries with the highest mortality for four different community-level prognostics — none, integrated management of childhood illness (IMCI), combined IMCI and pulse oximetry with 70% sensitivity (PO1) and combined IMCI and pulse oximetry with 85% sensitivity (PO2). DRC, Democratic Republic of the Congo.

Full size image

The predicted mortality across the 15 countries under different types of prognostic scenarios are shown in Figure 2b. In comparison to a baseline scenario of no prognostic tool (modelled as a prognostic with a sensitivity and specificity of 50%), IMCI is predicted to have a small incremental impact on mortality. This is largely driven by the fact that the sensitivity of IMCI in identifying severe cases is only 50–60%, and hence not much greater than 'chance'. By contrast, we predict that distribution of pulse oximetry to affected communities could result in much greater reductions in mortality. This is driven by both the higher sensitivity of pulse oximetry (65–75% for PO1, 80–90% for PO2) compared with IMCI (50–60%) and our assumption that there would be a higher adherence rate to PO1 and PO2 (80–90% for severe cases) compared with IMCI (60–70% for severe cases). The countries with the poorest rates of community-based treatment have the greatest reduction in mortality when only IMCI is considered (compared with no prognostic tool); whereas for PO1 and PO2, the countries with the highest incidence of disease are predicted to have the greatest reduction in mortality compared with no prognostic tool used.

The reduction in mortality under IMCI, PO1 and PO2 compared with the absence of a prognostic tool can be translated directly into estimates of deaths averted per 1,000 children at risk. Our estimates of the number of deaths averted by PO1 and PO2 is substantially higher than those estimated to be averted by IMCI, with the highest impact per 1,000 children in Somalia, Mali and Niger (Fig. 3a). Although the effect of IMCI on deaths averted is small relative to pulse oximetry, there is still a benefit compared with the absence of a prognostic tool. The estimated impact of pulse oximetry is more apparent when translated to absolute values scaled to country-specific under-5 population sizes (Fig. 3b). In absolute terms, the introduction of pulse-oximetry devices is estimated to result in the greatest annual reductions in pneumonia deaths in India (75,500 deaths averted per year for PO2) and in Nigeria (15,400 deaths averted per year for PO2) owing to their large under-five populations (128 million and 27 million, respectively). Collectively, we estimate that the implementation of PO1 (the more conservative estimate of IMCI and pulse-oximetry sensitivity) could avert 103,000 (90% range = 77,000–135,000) deaths annually across the 15 countries with the highest burden. For PO2, this increases to 148,000 (90% range = 112,000–193,000).

Figure 3: Deaths averted by prognostic tools across 15 countries.
figure 3

a, Estimated deaths averted per 1,000 children per year under integrated management of childhood illness (IMCI), PO1 (combined IMCI and pulse oximetry with 70% sensitivity) and PO2 (combined IMCI and pulse oximetry with 85% sensitivity) each compared with a baseline of no prognostic. b, Estimated absolute number of deaths averted by PO1 and PO2 compared with IMCI alone. Error bars show 90% range from the sensitivity analysis. DRC, Democratic Republic of the Congo.

Full size image

A key aim of improved prognostic tools is to increase the number of patients with severe disease receiving correct hospital referral. Using Nigeria as a country-level example, we estimated that the proportion of people with severe cases receiving hospital referral could increase by 44% by implementing PO1 or 62% by implementing PO2. We also estimated a substantial reduction in incorrect treatment — with the number of people with severe disease receiving community-based care alone (under treatment) decreasing by 19% (PO1) and 25% (PO2). However, a small increase in number of people with non-severe cases who receive hospital referral (over-treatment) was also predicted (from 3.99% of cases to 5.08% for both PO1 and PO2); this is due to the assumption that higher adherence is associated with a severe prognosis made by PO1 or PO2 compared with IMCI alone.

Increasing the sensitivity of the prognostic tool — which is assumed from adding pulse oximeters to existing IMCI — only has a substantial effect if other aspects of the health system are functioning at reasonable levels. Using Nigeria as a country-level example, we identified four key variables that determined the additional impact of a pulse oximeter — the availability of amoxicillin, hospital care, oxygen within the hospital and the prognostic tool.

The reduction in mortality is slightly more sensitive to the availability of community-based care than the availability of hospital-based care (Fig. 4a). This is due to two assumptions: if a person is referred to hospital, but is unable to access hospital care, then community-based care may instead be accessed; and community-based care includes the provision of a full course of amoxicillin. Therefore, if hospital-based care is unavailable, cases will probably receive amoxicillin instead, and a proportion of these will be cured by amoxicillin. Providing a more sensitive prognostic tool only had a substantial impact on mortality when the availability of hospital care was greater than 20%, when the oxygen availability exceeded 60% and when the prognostic tool was available in more than 60% of communities (Fig. 4a). We further investigated the impact of reduced oxygen availability on both deaths averted and cost-effectiveness (Fig. 4b) and found that oxygen availability (parameterized as the hospital cure rate) needs to be at least 60% for deaths to be averted by PO2 compared with IMCI alone. For cost-effectiveness to be less than US$40 per DALY averted, oxygen availability should exceed 70% (Fig. 4b).

Figure 4: Sensitivity analyses.
figure 4

a, The relationship between prognostic scenario parameters, prognostic sensitivity and the estimated annual pneumonia deaths per 1,000 children under 5 under varying prognostic scenarios: 1) the availability of amoxicillin (community-based care); 2) the availability of hospital care; 3) the availability of the prognostic tool; and 4) the availability of oxygen (hospital care). All other parameters were fixed at their central values. b, The impact of oxygen availability on cost-effectiveness and deaths averted, assuming a combined integrated management of childhood illness and pulse oximetry sensitivity of 85% (PO2). All scenarios are modelled using country-specific parameters for Nigeria. DALY, disability-adjusted life year.

Full size image

PO1 and PO2 are assumed to have both higher sensitivity and higher adherence to severe prognosis than IMCI alone. To assess the relative contributions of these two parameters, the change in incidence of cases in each of the four treatment states compared with IMCI was calculated for five scenarios: with the increase in adherence to a severe prognosis only; the increase in sensitivity for PO1 only; the PO1 combination of increased adherence and sensitivity; the increase in sensitivity for PO2 only; and the PO2 combination of increased adherence and sensitivity (Supplementary Fig. 3). We found that increasing each parameter alone had a substantial effect on the incidence of deaths and treated cases. Increased adherence to a severe prognosis alone caused more non-severe cases to be incorrectly treated owing to the poor sensitivity of IMCI that caused some non-severe cases to be given a prognosis of severe — these are then treated because of the higher adherence. The increase in severe prognosis adherence resulted in a small increase in incorrect treatment for non-severe cases (relative to the total number of non-severe cases), but a substantial increase in correct treatment for severe cases (relative to the total number of severe cases). The combination of increased adherence and prognostic sensitivity had the largest impact on the correct treatment of severe cases.

Across all the countries studied, community-based care costs (US$0.56–3.70 per course of amoxicillin15) are small in comparison to the corresponding hospital costs ($6.44–130.34 for a 7-day inpatient stay15). The estimated cost of the intervention itself (approximately $165 per 1,000 children per year) is also estimated to be lower than the additional health-care costs in most of the countries. As such, overall increases in health-care costs under pulse oximetry are largely associated with higher hospital referral rates for severe pneumonia cases.

The cost-effectiveness of implementing pulse oximetry in the 15 countries with the highest burden is shown in Figure 5. PO1 and PO2 are most cost-effective in Niger ($3.72 and $2.97 per DALY, respectively), the Democratic Republic of the Congo ($6.81 and $4.81 per DALY, respectively) and Ethiopia ($6.57 and $5.00 per DALY, respectively), partly driven by the comparatively lower costs of hospital care in those countries. For comparison, an insecticide-treated mosquito net (ITN) to prevent malaria is estimated to cost between $5 and $31 per DALY averted19, whereas HIV antiretroviral therapy has been estimated to cost upwards of $150 per DALY averted20.

Figure 5: Cost-effectiveness of prognostic tools.
figure 5

Estimated cost-effectiveness (US$ per disability adjusted life year (DALY)) of PO1 (combined integrated management of childhood illness (IMCI) and pulse oximetry with 70% sensitivity) and PO2 (combined IMCI and pulse oximetry with 85% sensitivity) compared with IMCI in the 15 countries with the highest burden of pneumonia. The numbers indicate the median estimate whereas bars represent the 90% range. DRC, Democratic Republic of the Congo.

Full size image

Discussion

Using a simple model that links care pathways to the progression of pneumonia in young children, we predict that a combination of pulse oximetry with current IMCI guidelines has the potential to avert up to 148,000 deaths per year in the 15 countries with the highest burden of pneumonia across Africa and Asia, under the assumption that there is more than 90% prognostic tool and supplementary oxygen availability. This equates to about one-sixth of all deaths owing to community-acquired pneumonia in the developing world. For comparison, it has been estimated that complete elimination of low birth weight would prevent 25% of pneumonia deaths in developing countries, with a similar proportion prevented by eliminating malnutrition21. Analysis of the impact of the pneumococcal vaccine for infants, PCV10, predicted that the vaccine has the potential to directly avert around 262,000 deaths in under-5s across 72 countries22. The relative ease of implementation of a pulse oximetry-based intervention (even with the assumption of perfect availability) compared with the elimination of low birth weight or malnutrition makes it an important candidate for an intervention against pneumonia in resource-poor settings.

On top of the large reduction in deaths, we predict that the addition of pulse oximetry to IMCI has the potential to increase the correct treatment of severe cases by an estimated 44%. When modelling the effect of PO1 and PO2 compared with IMCI, we increased two key parameters to simulate the implementation of pulse oximetry. These were prognostic sensitivity (which was set to be higher for PO1 and PO2 than for IMCI) and adherence to a severe prognostic result (also higher for PO1 and PO2 than for IMCI). Both substantially contribute to an increase in the correct treatment of severe cases and thus the predicted reduction in pneumonia deaths. Sensitivity analysis showed that an increase in either of these characteristics alone has the potential to prevent deaths. These substantial burden reductions are explained by the relatively low sensitivity of IMCI for detecting severe cases (just 55% compared with a potential 70–85% for pulse oximetry combined with IMCI), and the very high burden of pneumonia in these 15 countries (910,000 deaths attributed to pneumonia in under-5s in 2010 (ref. 12).

The incremental cost-effectiveness of PO1 and PO2 over IMCI was found to be very low in 14 of the 15 countries (less than $30 per DALY averted). For reference, the gross domestic product (GDP) per capita across these 14 countries ranged from $400 to $3,000 in 2013. Compared with the cost-effectiveness of the distribution of PCV10, estimated to be $100 per DALY averted22, this seems to be remarkably favourable. However, the extra costs of providing oxygen support were not taken into account in the calculations of cost-effectiveness, owing to a lack of data on the availability of oxygen support at the country level. Including the costs in the analyses will decrease the cost-effectiveness. Nevertheless, we predict that when coupled with the additional costs of oxygen support, pulse oximetry will still compare favourably with the PCV10 vaccine. For example, a study in Papua New Guinea estimated the cost-effectiveness of improving oxygen support in this area (including oxygen concentrators and the provision of pulse oximeters) to be $50 per DALY averted23.

There were several limitations to our analysis. One of these was the lack of country-specific data to inform our parameter for access to hospital care. We assumed that 61% of people who were referred to hospital would access hospital care, based on data from a retrospective case review study of children with severe pneumonia in Tanzania24. More realistically, we know that the proportion of children reaching an appropriate health facility may vary significantly between countries and so having one single parameter for all countries could result in inaccurate estimates. However, our sensitivity analysis showed that the model is less sensitive to hospital access than community-based care access. Another limitation was a lack of data on the availability of oxygen support across the 15 countries and how it is distributed throughout the health system. Linked to this was our assumption that the hospital systems in each country were substantial enough to support the extra cases that would be hospitalized. More field data on the hospital systems in each country is required to inform and expand the model to appropriately address these limitations. Nevertheless, it is clear that for any new prognostic to have impact there is the need to also invest in strengthening the existing primary and tertiary health-care facilities so that appropriate care is provided to those that are referred.