Objective:

To report on a new modelling approach developed for the assessing cost-effectiveness in obesity (ACE-Obesity) project and the likely population health benefit and strength of evidence for 13 potential obesity prevention interventions in children and adolescents in Australia.

Methods:

We used the best available evidence, including evidence from non-traditional epidemiological study designs, to determine the health benefits as body mass index (BMI) units saved and disability-adjusted life years (DALYs) saved. We developed new methods to model the impact of behaviours on BMI post-intervention where this was not measured and the impacts on DALYs over the child's lifetime (on the assumption that changes in BMI were maintained into adulthood). A working group of stakeholders provided input into decisions on the selection of interventions, the assumptions for modelling and the strength of the evidence.

Results:

The likely health benefit varied considerably, as did the strength of the evidence from which that health benefit was calculated. The greatest health benefit is likely to be achieved by the 'Reduction of TV advertising of high fat and/or high sugar foods and drinks to children', 'Laparoscopic adjustable gastric banding' and the 'multi-faceted school-based programme with an active physical education component' interventions.

Conclusions:

The use of consistent methods and common health outcome measures enables valid comparison of the potential impact of interventions, but comparisons must take into account the strength of the evidence used. Other considerations, including cost-effectiveness and acceptability to stakeholders, will be presented in future ACE-Obesity papers. Information gaps identified include the need for new and more effective initiatives for the prevention of overweight and obesity and for better evaluations of public health interventions.

Introduction

There is now a widespread awareness of the problem of childhood and adolescent obesity^{1, 2} and many governments are now seeking to invest in prevention and management programmes. Concurrently, there is also an increasing recognition of the need for health investments to be informed by the best available evidence of effectiveness and cost-effectiveness.^{3, 4, 5} Unfortunately, for the prevention of childhood obesity, the traditional evidence base of intervention effectiveness trials is very small in volume (fewer than 25 studies), very narrow in approach (mainly primary school programmes), shows very limited impact,^{6, 7} and evidence of cost-effectiveness is almost non-existent.^{8, 9} New approaches are urgently needed to fill this gap in the evidence base so that decision-makers have some evidential basis for making policy and funding decisions beyond the usual drivers of historical precedence, potential political gains and the lobby power of vested interests.

In recognition of the need for evidence-based policy making for obesity prevention, the Department of Human Services in Victoria, Australia, commissioned the assessing cost-effectiveness in obesity (ACE-Obesity) project in 2004. The aim of the project was to assist state and national policy-makers by providing evidence of the effectiveness and cost-effectiveness of selected obesity prevention interventions, particularly among children and adolescents.

The methods for the ACE-Obesity project, along with previous ACE studies in cancer, heart disease and mental health,^{4, 10, 11, 12} draw upon the theories of priority setting¹³ and combine technical rigour with due process. Technical rigour is obtained by using the best available epidemiological and economic data when analysing the effectiveness, population impact, costs and cost-effectiveness of interventions and by using a standardized evaluation protocol to ensure transparency and comparability of outcomes across interventions. Due process is achieved by involving stakeholders in a project working group, taking into account '2nd stage filter' criteria which are the broader considerations that are important in decision-making but less amenable to quantification (strength of evidence, acceptability to stakeholders, feasibility of implementation, sustainability, impact on equity as well as potential positive and negative side-effects^{5, 13}) and seeking consensus decisions after informed discussion and debate.

For previous ACE projects, there was sufficient trial evidence available for the technical analyses to model interventions for the Australian context. However, to evolve the previous ACE methodologies for use in childhood and adolescent obesity posed several substantial challenges: the limited trial data meant that other forms of evidence would have to be used; models for assessing the impact of behaviour changes on energy balance and weight change would have to be developed, and; methods for extrapolating reductions in body mass index ('BMI savings') in childhood into reductions in disability-adjusted life years ('DALY savings') in adulthood would also have to be developed. This work therefore involved the development of an integrated series of new methodologies for the obesity context that could be widely applied across many different obesity intervention scenarios. This work represents a significant advance on previous cost-effectiveness work in adults and children^{8, 14} because it includes public health interventions without randomized controlled trial evidence, assesses interventions using common health outcomes (BMI and DALYs) and includes the expected impact on almost all diseases that have been shown to be causally related to obesity.¹⁵

It is only when all interventions are assessed using common health outcomes that fair comparisons can be made across interventions and with interventions for other disorders. This has not been possible before for obesity and was only achievable through modelling as BMI and obesity-related disease outcomes are not routinely measured when assessing the effectiveness of interventions. Although this modelling has been done using Australian data for demographics, burden of disease, BMI distribution and intervention uptake the work could be readily adapted for use in other countries by keeping the same structure but using country-specific data for these inputs.

The purpose of this paper is twofold: to provide an overview of the ACE-Obesity approach and associated methods for calculating the likely impact of interventions on BMI and DALYs and, as an example, to provide the health benefit results for 13 obesity interventions modelled for the Australian population of children and adolescents. An assessment of the strength of evidence (level and quality) used in the calculations of health benefit is also reported.

Methods

Overview of ACE-Obesity aims and process

The ACE methodology aims to help decision-makers (mainly within governments) to set priorities by combining technically rigorous analyses with due process. The question posed by the ACE-Obesity project was 'What are the best options towards which state and national resources should be directed to reduce unhealthy weight gain in children and adolescents in Australia?' Although the project was funded by the Victorian State Government, it used the target population of all Australian children and adolescents aged 5–19 years. The reference year for all calculations was 2001 (latest year for which key data were available) and an annual discount rate of 3% was used for calculating the present value of the projections.³

A working group was established of representatives from state and federal health departments and other government agencies and departments, representatives from key non-governmental organizations (including a health consumer organization), and academics with particular areas of expertise. They met nine times over the 2 years of the project and provided substantial input into: the selection of the interventions to be scoped and then taken to full analyses; the methodologies developed for the modelling; the assumptions involved in the models, including uncertainty parameters; and the judgements about the 2nd stage filter criteria for each intervention.

Interventions

A list of 22 potential intervention areas was drawn from the literature, existing initiatives and programmes and the Australian Government's strategic plan for obesity.¹⁶ As part of the ACE-Obesity project we had the resources to evaluate the effectiveness and cost-effectiveness of 13 different obesity interventions in children and adolescents. Priority was given to public health interventions that met pre-agreed selection criteria; two of the most significant being (i) relevance to current policy decision-making and (ii) availability of evidence of effectiveness/efficacy to support the analyses. Judgements about which interventions to include were made by the project working group and supported by 'scoping reports' prepared by the research team. Although there was very little evidence to support some of the included interventions, they were included owing to their importance to current policy decision-making (e.g. Active After School Communities). Some clinical interventions were included for benchmarking purposes (e.g. Laparoscopic adjustable gastric banding). Intervention areas were most commonly excluded because a particular intervention could not be clearly described (e.g. active recreation), owing to lack of evidence (e.g. interventions in early child care) or because several interventions from that setting (e.g. schools) were already included.

The interventions chosen for analysis by the project working group are shown in Table 1. Also shown is the broad intervention setting, the target population for the intervention and the likely number of participants based on Australian population figures in 2001 and likely take-up rates.

Table 1 - Interventions chosen for analysis (grouped by setting), target population and strength of evidence (definitions in Table).

Full table

Assessment of health gain

DALYs saved over the child's lifetime was chosen as the measure of health gain for the ACE-Obesity project. This measure was chosen because it captures both morbidity and mortality effects and because baseline information on health status is available for Australia.^{43, 44} It also allows comparison with obesity prevention interventions in adults, as well as with interventions for other conditions for which cost-effectiveness ratios have been calculated using the ACE approach.^{4, 10} We also calculated the total age-specific BMI units (kg/m²) that could be saved by the intervention.

The first step was to calculate the BMI units (kg/m²) that the intervention could save for the individual and the population and the second step was to convert this post-intervention BMI units saved to lifetime DALYs saved.

Measurement of BMI units saved

Where possible, BMI change post-intervention was determined directly from trial data. Where there was more than one relevant (and similar) trial, we attempted to use meta-analysis to determine the average BMI benefit.⁴⁵ In practice, the degree of heterogeneity in intervention design (e.g. 'Multifaceted school-based interventions') prevented this and analyses were more often based on the results of one trial.

Where controlled trials were not available, effectiveness was modelled using the best available evidence and/or plausible assumptions. We make all of our assumptions explicit in reporting of the effectiveness and cost-effectiveness results in the individual intervention papers. For some interventions only changes in behaviour (e.g. an increase in physical activity) were reported in the trial or in the programme evaluation. Examples of these interventions include 'Reduction of TV advertising', where the randomized controlled trial only reported changes in food and drink choice and the 'Walking School Bus', where only frequency and duration of walking was reported for those participating in the intervention. To enable us to evaluate these interventions, we modelled the relationship between behaviour change, energy balance and BMI using the best available evidence. The method for this is described below.

Behaviour change to energy imbalance

The logic model for the pathways through which behaviour changes influence body weight has previously been published.⁴⁶ Interventions that change energy intake operate through changes in either the weight (g) or the energy density (kJ/g) of food and beverages consumed (or both). Interventions that change energy expenditure are assumed to operate through changes in sedentariness or changes in physical activity (or both).

There is now substantial experimental evidence that changing the energy density of foods and beverages changes total energy intake because there is little or no compensation through change in the total weight of food eaten.^{47, 48} This is supported by monitoring data in Australian children which showed that the 13% increase in energy intake from 1985 to 1995 was owing to the 15% increase in energy density with no changes in the weight of food eaten.⁴⁹

We used the Australian 1995 National Nutrition Survey (NNS95)⁵⁰ to determine the weight and energy density of the total diet (excluding water) for children 2–18 years from 'core' and 'non-core' foods and beverages according to the Australian Guide to Healthy Eating.^{51, 52} We assumed that any reductions in non-core food consumption (e.g. owing to reduced promotion of the products) were replaced by an equivalent weight of core foods so that the total weight of food was not altered. The same assumptions were made for beverages, but the impact on BMI was based on data from Ludwig et al.⁵³

Unless there were published trial data on the changes of physical activity or sedentary behaviours on weight or BMI, we modelled the impact of such interventions using the published energy costs of activity.⁵⁴ The metabolic equivalent values (METs) for an activity (such as walking) were subtracted from the MET value for resting quietly (1.0) to give the net MET difference for modelling.

Energy imbalance to weight and BMI

Once an energy deficit had been determined, the next step was to model its impact on changes in weight using validated coefficients for such calculations,⁴⁶ whereby a 10% change in energy balance results in a 4.5% change in body weight (95% confidence interval 3.8–5.1).⁴⁶ We used the data from the NNS95⁵⁰ to translate mean changes in weight to mean changes in BMI, assuming no change in height.

Modelling of DALYs saved

We calculated the DALYs saved owing to the interventions as the difference in future mortality and morbidity outcomes between a baseline scenario (base case), which represents current practice, and the intervention option. Differences in mortality and morbidity were based on changes in the age-specific BMI distribution of the target population over their remaining life span. To do this we took all children aged 5–19 years from the NNS95 as the cohort of children for the model, since data on measured BMI are available and representative of the Australian population in 1995.⁵⁰ For all analyses BMI values were log-transformed and 6 years of cohort effect were added to approximate 2001 BMI values. We calculated future predicted BMI values for this cohort of children in 5-year increments to 100 years of age by adding age effects to the values from 2001. We calculated both the cohort and age effects from serial cross-sectional studies of BMI in children and adults in Australia using multiple linear regression. These calculations assume that the relative contribution to BMI of year of birth (cohort) and age remain constant over time. These values of BMI represent the base case.

For the intervention scenario, we adjusted the base case BMI values in 2001 by subtracting the average BMI reduction attributable to the intervention in the relevant target group. Thus, for an intervention aimed at all 5–9 year olds that achieved a reduction in BMI of 0.2 kg/m² all 5–9 year olds in the cohort have 0.2 subtracted from their 2001 BMI values. If the intervention was targeted at obese children, only those children within the cohort that fell into that category would have 0.2 subtracted. We then log-transformed the BMI values and the projections into the future were recalculated by adding the age effects as described above. These values of BMI represent the intervention option.

Potential impact fraction

We determined the impact of the change in the BMI distribution on mortality and morbidity from the potential impact fraction (PIF).⁵⁵ The PIF is the proportional change in expected disease or death, that is, attributable to a change in exposure to the risk factor in the population. It is calculated from the proportion of children in each BMI category (e.g. 21–21.99, 22–22.99 and so on) in the base case and with the intervention, and from the relative risk (RR) of disease for the specific BMI category compared to a reference category of BMI 21.^{15, 55} BMI values below 21 are included in the reference category of BMI 21 (as used by the World Health Organization for their Global Burden of Disease Study).

The diseases for which PIFs were calculated are ischaemic heart disease, ischaemic stroke, hypertensive heart disease, type 2 diabetes, osteoarthritis, endometrial cancer, colon cancer, post-menopausal breast cancer and kidney cancer. These diseases were included because: (i) they have been shown to be causally related to obesity;¹⁵ (ii) continuous RRs are available (i.e. per 1 kg/m² increase in BMI);¹⁵ and (iii) they are included in the 2003 Australian and/or 2001 Victorian Burden of Disease Study estimates of disease risk owing to high BMI.⁴⁴ PIFs were calculated in Stata (Intercooled Stata, version 8.0, StataCorp) by sex and 5-year age group over the remaining lifespan of the cohort.

The RR estimates¹⁵ were applied to both deaths and disease prevalence and assume that: (i) excess disease risk occurs only at BMIs >21 kg/m² and only from 25 years of age;¹⁵ (ii) shifts in the population distribution owing to obesity interventions do not lead to an increase in the prevalence of underweight (BMI<18.5 kg/m²) which has its own health risks;¹⁵ (iii) RR estimates are not changing over time; (iv) risks for the 30–44 year age group apply to 25–29 year olds; (v) RRs are constant within a given age group; and (iv) the risk of disease is exponentially related to BMI.¹⁵ Disease risks for underweight (BMI <18.5 kg/m²) are not included.

The BMI to DALYs model

To determine the change in DALYs resulting from an intervention deterministic Markov modelling techniques are used. The model takes the current prevalent cohort of children and adolescents (age 5–19 years) in 2001 and follows them for their remaining life span until death (or age 100 years). New cases are not added. The model was constructed in Microsoft Excel with some inputs (PIFs, projected mortality rates) calculated in Stata.

The backbone of the model is a separate cohort life table for each sex (male, female) and 5-year age group (5–9, 10–14 and 15–19 years), giving six life tables in total for the base case. These six tables, constructed in Excel 2000, are repeated for the intervention scenario and the difference between the intervention and base case gives the DALYs saved owing to the intervention.

Base case (current practice) scenario

The base case assumed continuation of current trends in mortality and BMI distributions and, implicitly therefore, continuation of past trends in practice of obesity prevention and treatment. The years of life lived in the base case were calculated from projected Australian all-cause (total) age-specific mortality rates.⁵⁶ Projections were based on past mortality trends (1979–2002) with no explicit modelling of the effect of changes in risk factors and assumed continuation of trends to 2031, after which mortality rates were kept constant.

Years of life lived were adjusted for disease-related disability for each sex and age, determined from the Victorian Burden of Disease Study for 2001.⁴⁴ Total prevalent years lived with disability (YLD) for each sex and age group were calculated as prevalence disability weight and were converted to rates by dividing by Victorian population figures. Total mortality and disability rates were extrapolated from 5-year to 1-year age groups for input into the life tables using the function (linear, exponential or polynomial) that gave the best R²-value.

Intervention scenario

For the intervention scenario we calculated the mortality and prevalent YLD rates for obesity-related diseases as the rate in the base case (1-PIF) for each age and sex category. For most diseases it was assumed that current prevalent YLD rates are a good approximation of future rates. However, for ischaemic heart disease, ischaemic stroke and type 2 diabetes, trends in incidence were accounted for to 2031, after which they were kept constant.⁵⁶ The trends show decreasing rates of prevalent YLD for ischaemic heart disease and ischaemic stroke, but increasing rates for type 2 diabetes, largely owing to expected increases in the prevalence of obesity over the projection period.⁵⁶ After applying the PIFs we recalculated years of life lived and disability-adjusted years lived. The difference between the intervention scenario and the base case gives the years of life saved and DALYs saved by the intervention.

Targeted interventions

For interventions targeted at the overweight or obese group of children the average population mortality and YLD rates used in the BMI to DALYs model would underestimate the true disease experience of these higher risk groups. Therefore, correction factors were calculated for: (i) overweight or obese children; and for (ii) obese children separately by disease, sex, 5-year age group and cohort (aged 5–9, 10–14 and 15–19 years in 2001). This was done by determining the proportion of the disease burden in the whole population at each age that is contributed by the top percentiles of the BMI distribution that correspond with the prevalence of obesity and/or overweight in children targeted for intervention. For example, if the prevalence of obesity in 5- to 9-year-old children was 10% we calculated at each successive age the proportion of disease burden owing to elevated BMI that occurs in the top 10% of the BMI distribution. If in this example we found that 25% of the total amount of disease attributed to BMI occured in the top 10% of the BMI distribution, the correction factor was 25/10=2.5. In other words, children in the top 10% of the BMI distribution will experience 2.5 times the rate of the population average disease rates and treatment costs. When applied in the model the correction factors for the overweight or obese group resulted in an increase in the DALYs saved of 92–250% depending on the cohort, sex and BMI reduction tested.

Economic analyses

Economic analyses were also conducted and whereas the results are not shown here, brief mention is made of the methods to provide a complete picture of the ACE-Obesity approach. With each of the specified interventions, costs were estimated based on a societal perspective and with the intervention under 'steady-state conditions' (i.e. fully implemented and operating in accordance with its efficacy potential). Detailed intervention pathway analysis was used to specify all steps in an intervention and the probability of associated resource use. Unit costs were sourced from the most accurate sources for the 2001 reference year. Intervention costs were assessed as additional expenditure (savings) against current practice. Net incremental cost-effectiveness ratios were calculated as the 'net cost per DALY saved', following consideration of cost-offsets or the savings in health care costs arising from a reduction in obesity-related diseases as a consequence of the intervention.

Uncertainty analysis

We used @RISK software (version 4.0, Palisade Corporation) to conduct simulation modelling (with Monte Carlo sampling) to calculate 95% uncertainty ranges around the benefits, costs and cost-effectiveness ratios. The probability distributions around the input variables were based on the range of parameter values and/or standard errors obtained from the literature, together with expert advice on the likely scenarios under Australian conditions.

In the BMI to DALYs model, there is uncertainty around many of the input values, such as the RRs of disease, mortality rate projections and BMI projections. Ideally, we would have included uncertainty around all inputs in the @RISK analyses. However, this is a difficult and time-consuming task and thus, to date, we have only conducted simple one-way uncertainty analyses in the BMI to DALYs component of the modelling. Consequently, the only uncertainty around the resultant DALYs saved for each intervention is due to the uncertainty around the impact of each intervention on BMI.

Strength of evidence

When assessing the strength of evidence used to calculate the health benefit, it became clear that the existing ACE approach¹⁰ based on the grading of the level⁵⁷ and quality of evidence into three categories (sufficient evidence, limited evidence, and inconclusive evidence, as set out in the left hand column of Table 2) did not work very well for obesity prevention interventions. Thus, a new classification system was developed (Table 2), which combined the traditional classification of evidence based on epidemiologic study design, with other types of evidence that would not ordinarily be captured, for example, parallel evidence and epidemiological modelling to BMI based on a mix of evidence types. This revised classification draws on the work of Hawe and Shiell 1995⁵⁸ and Swinburn et al. 2005⁵ and also reflects aspects of other evidence frameworks.^{59, 60, 61}

Table 2 - Classifying the strength of the evidence – based on study design⁵⁷ and quality.

Full table

Results

Results for the assessment of health benefit and strength of evidence for the 13 interventions analysed are shown in Tables 1 and 3. The intervention with the biggest impact on an individual's BMI is the clinical intervention 'Laparoscopic adjustable gastric banding' for morbidly obese adolescents (Table 3). However, the intervention with the biggest population impact is the public health intervention 'Reduction of TV advertising of high fat and/or high sugar foods and drinks to children'. Although the individual impact of this intervention is relatively small (Table 3), the number of children affected is large (Table 1), resulting in savings of around 400 000 total BMI units or 37 000 total DALYs (Table 3).

Table 3 - Impact of the interventions on BMI (post-intervention) and DALYs saved (over the child's lifetime).

Full table

Although there are multiple considerations in choosing an eventual portfolio of interventions to reduce childhood obesity, some of the factors can be displayed graphically to give a sense of the interplay between them. An example is shown in Figure 1 where the strength of the evidence (from Table 1) for the modelled interventions is plotted against its estimated population impact (from Table 3). Options in the top left hand corner, provide the biggest population impact with the greatest strength of evidence and, therefore, greater certainty of effect.

Figure 1.

Total DALYs saved versus strength of the evidence base (definitions in Table 2).

Full figure and legend (73K)

Discussion

Investing in the reduction of obesity in children and adolescents means setting priorities. Ideally, the programmes, policies and services to be prioritized into a portfolio of interventions should be based on the best evidence available on the likely costs, the likely effectiveness, the level of certainty (strength of evidence) and other considerations important in decision-making (impact on equity and so on).^{4, 5} The ACE-Obesity approach aims to provide this information to decision-makers and, in this paper, we have provided an overview of the approach and some of the key methodologies along with illustrative health benefit (effectiveness) results from the modelling as applied to 13 potential interventions for the Australian population of children and adolescents.

At this stage, the ACE-Obesity methods and processes may be analogous to the early modelling of the global burden of disease in the 1990s.⁶² Those results had a significant effect on reorienting global attention onto the major contributors to the overall population disease burden. Since then, that modelling methodology has become much more sophisticated to include risk factors and attributable burdens, the evidence base to support the modelling has improved, and the modelling has been used to inform priorities in smaller jurisdictions such as at a state level.^{44, 63} The results of this ACE-Obesity modelling (particularly the effectiveness, costs, cost-utility and 2nd stage filters) will help to paint the broad picture of priorities for intervention and we would expect this picture to become more fine-grained and more contextualized over the coming years as the methods evolve, the evidence to include in the models improves, and other intervention scenarios are modelled. The ACE approach will help to bridge the gap between existing decision-making for resource allocation and policy formulation and the ideal situation of being able to choose proven interventions based on the best evidence possible.

Some general conclusions from this first set of ACE-Obesity analyses can be made. Firstly, unless the individual impact of an intervention is very large, such as for Laparoscopic adjustable gastric banding, the number of children and adolescents impacted by the intervention is a critical factor in determining population impact and is one of the reasons that the regulations to reduce food and drink advertising, which affect all children aged 5–14 years, had a bigger estimated effect compared to, for example, programmes in schools which are dependent on the level of school and student uptake. Another general observation was that the individual impact of physical activity interventions such as the active transport programmes was smaller than nutrition interventions such as reducing fizzy drinks. This is not really surprising given that, for the average 8-year-old child, 10% of energy intake is equivalent to 450 ml of soft drink (just over one can) whereas 10% of energy expenditure is equivalent to 2.5 h extra walking.⁴⁶

The use of consistent methods and common health outcome measures enables valid comparison of the potential impact of the 13 interventions presented in this example (Table 3), but comparisons should take into account the strength of the evidence used. When we consider the strength of evidence alongside the size of the total DALY benefit (Figure 1), decisions on what interventions are likely to be effective when implemented become more complex. Decisions to invest in interventions in the top left hand corner of Figure 1 are less risky because the evidence supporting a high potential health gain is much stronger. Thus, the chance of success in improving population health is higher. But, these interventions on their own are unlikely to be sufficient to address the current levels of overweight and obesity, let alone to prevent the future predicted increase in overweight and obesity. Thus, some tough decisions will need to be made – and these are more risky because the return is less certain. For example, the evidence supporting the reduction in TV advertising to children is less strong. Although there is a randomized controlled trial supporting the link between advertising and unhealthy food choice, the relationship between food choice and BMI was modelled based on lower levels of evidence. Thus, whereas the potential health gain is high, the certainty that it will be achieved if the intervention is implemented is lower. For this reason, we recommend that, if implemented it (and all interventions that fall in the right hand section of Table 2 and Figure 1) should be accompanied by an appropriate evaluation plan and budget so that the size of the actual impact on BMI can be determined using a controlled study design (level II or III⁵⁷). Changes to the intervention can then be made, if needed, based on better quality evidence. If not shown to be effective, the resources should then be diverted appropriately.

Another consideration in setting priorities for obesity prevention is the appropriate mix of clinical/targeted interventions versus public health interventions. Clinical interventions, whereas more likely to have an immediate and larger impact on individuals, are insufficient to reverse the trend towards increasing BMI.⁶⁴ Real impacts on the problem will only occur if clinical approaches are complemented by public health prevention approaches that focus on the whole population with the aim of changing social norms and moving the entire distribution of BMI to the left.⁶⁴ The need to intervene across a range of settings and sectors should also be considered for maximum effect.⁵

Methodological advances

We chose to develop new methods rather than exclude all interventions that did not have randomized controlled evidence of a change in BMI, which would have reduced the project's usefulness to policy makers. We used the best available evidence to model from changes in behaviour to changes in BMI. We also developed a new evidence grading system based on the degree of confidence that we thought could be put in the results for interventions that relied on evidence of effectiveness from level IV study designs and other types of evidence (see right hand column of Table 2). Although both of these approaches increase the uncertainty around the benefit estimate we believe that this is preferable to allowing selective decision-making to continue where interventions with poorer quality evidence are implemented solely on the basis of political acceptability without consideration of their potential health gain or cost-effectiveness. But we reiterate that, if implemented, these interventions must be properly evaluated to confirm their effectiveness and to enable modifications to be made if needed.

Another methodological advance in this project was to model from post-intervention changes in BMI to DALYs saved over the child's lifetime using the best available evidence. DALYs are particularly useful because they allow comparisons to be made between interventions in children and adults as well as between interventions addressing other disorders or risk factors. The use of a utility measure such as the DALY also allows 'benchmarking' with existing pharmaceutical and medical interventions.

Another strength of the ACE-Obesity project methods described in this paper is the inclusion of future predicted trends in BMI, deaths and morbidity to 30 years in the future. Although adding uncertainty to the model, it improves its overall accuracy. Further, the treatment of BMI as a continuous rather than categorical variable (i.e. normal weight, overweight or obese) gives more valuable information on overall population impact of interventions and is consistent with the latest methodological advances made in assessing the burden of disease attributable to high BMI.¹⁵ Our use of simulation-modelling techniques to allow multi-way uncertainty analysis around the BMI benefit is another strength of our methods.

Limitations of the methods

The biggest limitation of the ACE-Obesity project methods is the assumption that 100% of the BMI benefit is maintained over the lifetime. This is unlikely, with the exception of the 'Laparoscopic adjustable gastric banding' intervention, as behavioural changes are difficult to maintain. Thus, the DALY benefit is likely to be lower than reported, unless implementation of multiple obesity prevention interventions had some kind of synergistic and multiplicative effect – which is currently unknown and untested. Also, if the intervention effort was maintained it could be postulated that the BMI benefit would be maintained as well. We tested the impact of the alternative assumptions that only 50 and 25% of the effect is maintained. The results indicate that the benefits are similarly reduced by approximately 50 and 25%, respectively. Although it is very likely that the size of the maintenance of the effect varies between interventions, there is insufficient evidence on which to base quantitative modelling. It is important to acknowledge that if all of the benefit is lost by the time the child reaches 25 years of age (the age at which disease risks begin to be included in the model) then the resulting DALYs saved will be zero.

This highlights another limitation of the model: that impacts on diseases in childhood and adolescence are not included, nor is any impact on quality of life, that is, independent of disease. However, this was a deliberate decision because there is a lack of reliable epidemiological evidence that any childhood conditions are causally related to obesity² or that BMI has an impact on quality of life, independent of disease. Other limitations of the ACE-Obesity project methods include the number of assumptions needed to translate from behaviour change to BMI to DALYs, as well as the inability to include uncertainty around all of these assumptions/estimates in our multi-way uncertainty analyses.

The lack of good quality evidence of effectiveness for many public health interventions, including some interventions that have already been widely implemented in Australia, was a significant limitation of the ACE-Obesity project. Although the reasons for not measuring BMI changes owing to an intervention are understandable (although not always well justified), the reasons for not including both pre- and post-intervention measurement of the targeted behaviour (e.g. physical activity or food choice), let alone an appropriate control group, are much less understandable and need to be addressed. In an era of greater demand for evidence-based policy-making and where the need to maximize the health gain achieved from limited resources is more apparent, it is imperative that funders of public health programmes ensure that what is implemented is properly evaluated. This information can then aid future decision-making.

Gaps in the knowledge base

As well as providing evidence of the health benefit likely to be achieved by various obesity interventions, the ACE-Obesity study has highlighted many information gaps that need to be addressed by further research. The most obvious is the need for new and more effective initiatives for primary prevention in particular. The expected gain from the current arsenal of interventions is unlikely to be sufficient to reverse the trend towards increasing levels of overweight and obesity. And any new or current interventions that are tested or implemented, need to be properly evaluated so that we can be confident that they actually achieve the desired impact when compared to current practice or to no intervention – be it an increase in physical activity levels, a decrease in consumption of energy-dense foods and drinks and/or a slowing of the increase in BMI. Impact on satisfaction levels or retrospective reporting of baseline behaviours is insufficient evidence on which to base funding decisions if the desire is to prove an impact on obesity or physical activity levels. Population-based cohort studies are also needed to properly assess the causal relationship between BMI in childhood and later health effects in children and adolescents, including type 2 diabetes and early signs of cardiovascular disease. The population prevalence of type 2 diabetes in children and adolescents also needs to be measured objectively and periodically to confirm the clinical and anecdotal evidence of its reported increase in recent years.^{2, 65}

Extrapolation of the health benefit results to other countries and health systems

In the ACE-Obesity study we used the Australian context to determine the likely health benefit (and costs). Therefore, care must be taken if trying to generalize the results to other countries. Some interventions may work differently in other countries, particularly behavioural interventions, owing to differences in lifestyle, culture, beliefs, education and health systems and current practice in obesity prevention. The impact on total BMI units and DALYs saved will also vary according to differences in population size and structure and different BMI distributions and disease rates between countries. Ideally, analyses should be repeated for other countries so that these factors can be taken into account before application in policy-making.

Conclusions

In conclusion, the results of these analyses provide valuable information on the health benefit likely to be achieved by 13 different interventions for the prevention of obesity in children and adolescents. Because we have used consistent methods and common health outcomes (BMI and DALYs), the impact of these interventions can be validly compared. However, conclusions drawn from these comparisons should take into account the strength of the evidence used in determining the health benefit. Other considerations that will be important and are to be presented in future ACE-Obesity papers are: the cost-effectiveness of the interventions and broader aspects that impact on real world decision-making, including acceptability to stakeholders, feasibility of implementation, sustainability, impact on equity and potential positive and negative side-effects.

References

1. International Food Information Council. Trends in obesity related media coverage. 2006 (accessed August 2006); Available from: http://www.ific.org/research/obesitytrends.cfm.
2. Lobstein T, Baur L, Uauy R. Obesity in children and young people: a crisis in public health. Obes Rev 2004; 5 (Suppl 1): 4–104. | Article | PubMed |
3. Gold MR, Siegel JE, Russell LB, Weinstein MC (eds). Cost-Effectiveness in Health and Medicine. Oxford University Press: New York, 1996.
4. Vos T, Haby MM, Magnus A, Mihalopoulos C, Andrews G, Carter R. Assessing cost-effectiveness in mental health: helping policy-makers prioritize and plan health services. Aust NZ J Psychiatry 2005; 39: 701–712.
5. Swinburn B, Gill T, Kumanyika S. Obesity prevention: a proposed framework for translating evidence into action. Obes Rev 2005; 6: 23–33. | Article | ChemPort |
6. Doak CM, Visscher TL, Renders CM, Seidell JC. The prevention of overweight and obesity in children and adolescents: a review of interventions and programmes. Obes Rev 2006; 7: 111–136. | Article | ChemPort |
7. Summerbell CD, Waters E, Edmunds LD, Kelly S, Brown T, Campbell KJ. Interventions for preventing obesity in children (Cochrane review). Cochrane Database of Systematic Reviews 2005; Art No.: CD001871.
8. Segal L, Mortimer D, Dalziel K. Risk factor study: how to reduce the burden of harm from poor nutrition, tobacco smoking, physical inactivity and alcohol misuse: cost-utility analysis of 29 interventions, executive report. Research Paper 2005 (1). Centre for Health Economics, Monash University: Melbourne 2005.
9. Wang LY, Yang Q, Lowry R, Wechsler H. Economic analysis of a school-based obesity prevention program. Obes Res 2003; 11: 1313–1324. | PubMed | ISI |
10. Carter R, Stone C, Vos T, Hocking J, Mihalopoulos C, Peacock S et al. Trial of Program Budgeting and Marginal Analysis (PBMA) to assist cancer control planning in Australia, Research Report 19. Centre for Health Program Evaluation: Melbourne, 2000.
11. Liew D, Lim SS, Bertram M, McNeil JJ, Vos T. A model for undertaking effectiveness and cost-effectiveness analyses of primary preventive strategies in cardiovascular disease. Eur J Cardiovasc Prevent Rehab 2006; 13: 515–522.
12. Haby MM, Carter R, Mihalopoulos C, Magnus A, Sanderson K, Andrews G et al. Assessing cost-effectiveness – mental health: introduction to the study and methods. Aust NZ J Psychiatry 2004; 38: 569–578.
13. Carter R. The Macro Economic Evaluation Model (MEEM): an approach to priority setting in the health sector (PhD thesis). Monash University: Melbourne, 2001.
14. Avenell A, Broom J, Brown TJ, Poobalan A, Aucott L, Stearns SC et al. Systematic review of the long-term effects and economic consequences of treatments for obesity and implications for health improvement. Health Technol Assess 2004; 8(21): 1–194. | ISI |
15. James WPT, Jackson-Leach R, Mhurchu CN, Kalamara E, Shayeghi M, Rigby NJ et al. Overweight and obesity (high body mass index). In: Ezzati M, Lopez AD, Rodgers A, Murray CJL (eds). Comparative Quantification of Health Risks: Global and Regional Burden of Disease Attributable to Selected Major Risk Factors. Geneva: World Health Organization, 2004. pp 497–596.
16. National Obesity Taskforce. Healthy Weight 2008 – Australia's Future – The National Action Agenda for Children and Young People and their Families. Department of Health and Ageing: Canberra, 2003.
17. Australian Government. Active After-School Communities Program. 2005 (accessed 20 August 2006); Available from: http://www.healthyactive.gov.au/internet/healthyactive/publishing.nsf/
    Content/initiatives-a.
18. Mo-suwan L, Pongprapai S, Junjana C, Puetpaiboon A. Effects of a controlled trial of a school-based exercise program on the obesity indexes of preschool children. Am J Clin Nutr 1998; 68: 1006–1011. | PubMed | ISI | ChemPort |
19. Sallis JF, McKenzie TL, Alcaraz JE, Kolody B, Hovell MF, Nader PR. Project SPARK. Effects of physical education on adiposity in children. Ann NY Acad Sci 1993; 699: 127–136. | PubMed | ChemPort |
20. Dwyer T, Coonan WE, Leitch DR, Hetzel BS, Baghurst RA. An investigation of the effects of daily physical activity on the health of primary school students in South Australia. Int J Epidemiol 1983; 12: 308–313. | PubMed | ISI | ChemPort |
21. Tamir D, Feurstein A, Brunner S, Halfon ST, Reshef A, Palti H. Primary prevention of cardiovascular diseases in childhood: changes in serum total cholesterol, high density lipoprotein, and body mass index after 2 years of intervention in Jerusalem schoolchildren age 7–9 years. Prev Med 1990; 19: 22–30. | Article | PubMed | ISI | ChemPort |
22. Manios Y, Moschandreas J, Hatzis C, Kafatos A. Evaluation of a health and nutrition education program in primary school children of Crete over a three-year period. Prev Med 1999; 28: 149–159. | Article | PubMed | ISI | ChemPort |
23. Foster GD, Wadden TA, Brownell KD. Peer-led program for the treatment and prevention of obesity in the schools. J Consult Clin Psychol 1985; 53: 538–540. | PubMed | ChemPort |
24. James J, Thomas P, Cavan D, Kerr D. Preventing childhood obesity by reducing consumption of carbonated drinks: cluster randomised controlled trial. Br Med J 2004; 328: 1237–1243. | Article |
25. Ebbeling CB, Feldman HA, Osganian SK, Chomitz VR, Ellenbogen SJ, Ludwig DS. Effects of decreasing sugar-sweetened beverage consumption on body weight in adolescents: a randomized, controlled pilot study. Pediatrics 2006; 117: 673–680. | Article |
26. Robinson TN. Reducing children's television viewing to prevent obesity: a randomized controlled trial. JAMA 1999; 282: 1561–1567. | Article | PubMed | ISI | ChemPort |
27. Gortmaker SL, Peterson K, Wiecha J, Sobol AM, Dixit S, Fox MK et al. Reducing obesity via a school-based interdisciplinary intervention among youth: Planet Health. Arch Pediatr Adolesc Med 1999; 153: 409–418. | PubMed | ISI | ChemPort |
28. DiPietro G, Hughes I. TravelSMART Schools – there really is a better way to go. 26th Australasian Transport Research Forum; Wellington, New Zealand; 2003. Available from: http://www.travelsmart.vic.gov.au/Web4/tsmart.nsf/.
29. VicHealth. Walking School Bus. 2006 (accessed 20 August 2006); Available from: http://www.vichealth.vic.gov.au/Content.aspx?topicID=23.
30. Victoria University. The Walking School Bus Program. Learnings from VicHealth's pilot program 2001. VicHealth: Melbourne, 2003.
31. Gorn G, Golberg M. Behavioural evidence of the effects of televised food messages to children. J Consum Res 1982; 9: 200–205. | Article |
32. McCallum Z, Wake M, Gerner B, Harris C, Gibbons K, Gunn J et al. Can Australian general practitioners tackle childhood overweight/obesity? Methods and processes from the LEAP (Live, Eat and Play) randomized controlled trial. J Paediatr Child Health 2005; 41: 488–494. | Article |
33. Flodmark CE, Ohlsson T, Ryden O, Sveger T. Prevention of progression to severe obesity in a group of obese schoolchildren treated with family therapy. Pediatrics 1993; 91: 880–884. | PubMed | ChemPort |
34. Israel AC, Stolmaker L, Andrian CAG. The effects of training parents in general child management skills in a behavioural weight loss program for children. Behav Ther 1985; 16: 169–180. | Article |
35. Kirschenbaum DS, Harris ES, Tomarken AJ. Effects of parental involvement in behavioural weight loss therapy for preadolescents. Behav Ther 1984; 15: 485–500. | Article |
36. Senadiak CSS. Rapid versus gradual scheduling of therapeutic contact in a family-based behavioural weight control programme for children. Behav Psychother 1985; 13: 265–287.
37. Chanoine JP, Hampl S, Jensen C, Boldrin M, Hauptman J. Effect of orlistat on weight and body composition in obese adolescents: a randomized controlled trial. JAMA 2005; 293: 2873–2883. | Article | PubMed | ChemPort |
38. Abu-Abeid S, Gavert N, Klausner JM, Szold A. Bariatric surgery in adolescence. J Pediatr Surg 2003; 38: 1379–1382. | Article |
39. Avinoah E. Laparoscopic gastric banding for morbidly obese adolescents. Surg Laparosc Endosc Percutaneous Tech 2004; 14: 350.
40. Fielding GA, Duncombe JE. Laparoscopic adjustable gastric banding in severely obese adolescents. Surg Obes Relat Dis 2005; 1: 399–405.
41. Hayr T, Widhalm K. Laparoscopic gastric banding in morbidly obese adolescents. Ann Nutr Metab 2003; 47: 427.
42. Widhalm K, Dietrich S, Prager G. Adjustable gastric banding surgery in morbidly obese adolescents: experiences with eight patients. Int J Obes 2004; 28: S42–S45. | Article |
43. Mathers C, Vos T, Stevenson C. The burden of disease and injury in Australia. AIHW cat. no. PHE 18. Australian Institute of Health and Welfare: Canberra 1999. (also available at) http://www.aihw.gov.au/publications/.
44. Public Health Group. Victorian Burden of Disease Study: Mortality and Morbidity in 2001. Department of Human Services: Melbourne, 2005.
45. Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Smith GD, Altman DG (eds). Systematic Reviews in Health Care: Meta-Analysis in Context 2nd edn. BMJ Publishing Group: London, 2001. pp 285–312.
46. Swinburn BA, Jolley D, Kremer PJ, Salbe AD, Ravussin E. Estimating the effects of energy imbalance on changes in body weight in children. Am J Clin Nutr 2006; 83: 859–863. | ChemPort |
47. Rolls BJ. The role of energy density in the overconsumption of fat. J Nutr 2000; 130: 268S–271S. | PubMed | ISI | ChemPort |
48. Rolls BJ, Roe LS, Meengs JS. Reductions in portion size and energy density of foods are additive and lead to sustained decreases in energy intake. Am J Clin Nutr 2006; 83: 11–17. | ChemPort |
49. Cook P, Rutishauser IHE, Seelig M. Comparable Data on Food and Nutrient Intake and Physical Measurements from the 1983, 1985 and 1995 National Nutrition Surveys. Commonwealth Department of Health and Aged Care: Canberra, 2001.
50. McLennan W. Australian Bureau of Statistics. Information paper: National Nutrition Survey, confidentialised unit record file, 1995. Australian Bureau of Statistics: Canberra, 1998.
51. Bell AC, Kremer PJ, Magarey AM, Swinburn BA. Contribution of 'noncore' foods and beverages to the energy intake and weight status of Australian children. Eur J Clin Nutr 2005; 59: 639–645. | Article | ChemPort |
52. Australian Department of Health & Ageing. Australian Guide to Healthy Eating. Australian Department of Health & Ageing: Canberra, 1998.
53. Ludwig DS, Peterson KE, Gortmaker SL. Relation between consumption of sugar-sweetened drinks and childhood obesity: a prospective, observational analysis. Lancet 2001; 357: 505–508. | Article | PubMed | ISI | ChemPort |
54. Ainsworth BE, Haskell WL, Whitt MC, Irwin ML, Swartz AM, Strath SJ et al. Compendium of physical activities: an update of activity codes and MET intensities. Med Sci Sports Exerc 2000; 32: S498–S504. | Article | PubMed | ISI | ChemPort |
55. Ezzati M, Lopez AD, Rodgers A, Vander Hoorn S, Murray CJ. Selected major risk factors and global and regional burden of disease. Lancet 2002; 360: 1347–1360. | Article | PubMed | ISI |
56. Vos T, Goss J, Begg S, Mann N. Australian Burden of Disease and Injury Study: Projected Health Care Costs Report. School of Population Health, University of Queensland and Australian Institute of Health and Welfare: Brisbane, 2005.
57. National Health and Medical Research Council. A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines. NHMRC: Canberra, 1999.
58. Hawe P, Shiell A. Preserving innovation under increasing accountability pressures: the health promotion investment portfolio approach. Health Promot J Aust 1995; 5: 4–9.
59. NHMRC National Breast Cancer Centre Psychosocial Working Group. Psychosocial Clinical Practice Guidelines: Information, Support and Counselling for Women with Breast Cancer. National Health and Medical Research Council: Canberra, 2000.
60. Loxley W, Toumbourou JW, Stockwell T. The Prevention of Substance use, Risk and Harm in Australia: A Review of the Evidence. The National Drug Research Institute and the Centre for Adolescent Health: Canberra, 2004.
61. Rychetnik L, Frommer M. A Schema for Evaluating Evidence on Public Health Interventions; Version 4. National Public Health Partnership: Melbourne, 2002.
62. World Bank. World Development Report 1993: Investing in Health. Oxford University Press: New York, 1993.
63. Ezzati M, Lopez AD, Rodgers A, Murray CJL (eds). Comparative Quantification of Health Risks: Global and Regional Burden of Disease Attributable to Selected Major Risk Factors. World Health Organization: Geneva, 2004.
64. Rose G. Sick individuals and sick populations. Int J Epidemiol 1985; 14: 32–38. | PubMed | ISI | ChemPort |
65. McMahon SK, Haynes A, Ratnam N, Grant MT, Carne CL, Jones TW et al. Increase in type 2 diabetes in children and adolescents in Western Australia. Med J Aust 2004; 180: 459–461. | PubMed |

Acknowledgements

We thank members of the ACE-Obesity working group for their input into the project: Michael Ackland (Deputy Chair), Bill Bellew, John Catford, Elizabeth Develin, Helen Egan, Bonnie Field, Tim Gill, John Goss, Robert Hall (Chair), Brian Harrison, Kellie-Ann Jolly, Mark Lawrence, Amanda Lee, Tony McBride, Karen McIntyre, Jan Norton, Anna Peeters, Melissa Wake and Rowland Watson. We also thank the other researchers who have worked on the project as part of their postgraduate studies: Jaithri Ananthapavan, Leah Galvin, Margaret MacDonald and Margaret Rumpf.