Introduction

Ecological or correlational studies using publicly available data may help identify possible external factors influencing health. However, they can be misleading as correlation does not mean causation. Associations found between groups may not reflect true associations between individuals in groups. Information on other important risk-modifying factors may not be available or may be ignored by the researchers. Limited statistical analysis may simply reinforce pre-existing ideas instead of really testing those ideas.

The choice of the main variables for consideration is inevitably dictated to a large extent by the hypothesis under review. However, the choice of covariates for consideration as risk-modifying factors will influence the effectiveness of the exploratory statistical analysis.

These problems are illustrated by two recent studies which used data for individual US states from publicly accessible US databases to test two different hypotheses for possible factors influencing attention deficit hyperactivity disorder (ADHD) prevalence in youth.

Malin & Till1 tested a hypothesis that ADHD prevalence is associated with the extent of community water fluoridation (CWF). Their introductory discussion considered possible chemical toxicants such as manganese, poly-chlorinated biphenyls, nicotine, mercury, arsenic, food additives and lead – exhibiting a preference for a chemical toxicity model. They did not include any of these as covariates in their statistical analysis but they did include naturally occurring fluoride levels and a measure of socio-economic status (household income) as the only confounding factors.

Huber et al.2 investigated a possible association of ADHD prevalence with altitude. This was based on a hypothesis that mild hypobaric hypoxia at higher altitudes can increase brain dopamine levels. Several studies have reported decreased brain ADHD dopamine levels in children and adolescents with ADHD and they hypothesised that US states at higher altitudes may have a lower prevalence of ADHD. They considered a range of potential social or psychological risk-modifying factors together with altitude, but did not consider CWF or potential chemical toxicants.

The Malin & Till1 study has been heavily promoted by groups campaigning against community water fluoridation. While there are a number of studies of cognitive deficits in areas of endemic fluorosis where drinking water fluoride concentrations are higher, mainly in China and India, this is the only published report of any negative psychiatric results from CWF. So it is not surprising the study is often cited in campaigns such as the recent petition by Connett3 to the US Environmental Protection Agency (EPA) and in the publications of researchers opposed to community water fluoridation – for example, Hirzy et al.4

In their response to the Connett petition3 the EPA judged5 that the Malin & Till1 study 'is not sufficient to ''corroborate'' neurotoxic health effects, as stated in the petition.' Malin & Till1 themselves stressed further work was necessary, particularly examining possible covariates, to test their hypothesis. The different association for ADHD prevalence found by Huber et al.2 endorses that recommendation.

This contribution extends the statistical analysis of Malin & Till1 to include as covariates state elevation and the range of social confounders that Huber et al.2 found to be statistically significant.

Methods

Data were taken from the sources used by Malin & Till1 and Huber et al.2 Initially, simple correlations between ADHD prevalence in 2011 and selected factors were considered. Multiple regression analysis was then carried out using the factors which had individually shown statistically significant correlations. The correlation and regression functions in the Microsoft Excel data analysis package were used.

The variables considered included extent of CWF in 1992 and median household income in 1992 considered by Malin & Till.1 Also included were variables identified as being independently significantly associated with ADHD prevalence by Huber et al.2 These were: mean state altitude; percentage of overweight/obese youth; percentage of low birth weight; percentage uninsured; percentage ever diagnosed with depression; percentage ever diagnosed with anxiety; percentage Hispanic; percentage Black, non-Hispanic; percentage other race; and percentage living in a two-parent household.

Other variables considered were home ownership%;6 education level (% with bachelor's degree in 2000);7 percentage of adults who smoked in 2013;8 percentage over 65 years of age in 20109 and longitude.10

Results

The covariates exhibiting independent and statistically significant (P <0.05) associations with ADHD prevalence in 2011 (Malin & Till)1 are listed in Table 1.

Table 1 Variables exhibiting statistically significant correlations (P < 0.05) with ADHD prevalence in 2011

A multiple regression analysis for ADHD prevalence in 2011 (Malin & Till)1 including the covariates CWF in 1992, median household income in 1992 and mean state elevation produced the results in Table 2.

Table 2 Multiple regression analysis for ADHD prevalence including CWF, median household income and mean state elevation as covariates

All the covariates in Table 1 were included in a multiple regression analysis and non-significant variables were removed one at a time until only statistically significant covariates remained. Only mean state elevation and per capita personal income in 2009 were statistically significant (see Table 3).

Table 3 Statistically significant covariates from multiple regression.

Discussion

There is a wide range of data sources which can be used in exploratory analyses. For a selected variable within a source one could choose data for different years. Malin & Till1 did this for their ADHD prevalence (2003, 2007, 2011) and fluoridation extent (1992, 2000, 2002, 2004, 2006, 2008) variables. Huber et al.2 selected ADHD data from 2007 and 2010 and matched the source year for the considered covariates where possible.

Although related (Pearson correlation coefficient 0.641) the ADHD prevalence figures used by Huber et al.2 for 2010 and by Malin & Till1 for 2011 appear to use different criteria. The mean of 2010 ADHD data used by Huber et al.2 was 30.46% with a range of 21.5–41.4%. The mean was 11.47% and the range 5.6–18.7% for the 2011 ADHD prevalence data used by Malin & Till.1 This illustrates one problem in choosing data and data sources for exploratory analysis. Selection of source year could be justified, to some extent, by the considered mechanisms. For example, perhaps older data for fluoridation extent may match more recent data for ADHD prevalence if the considered mechanism involved early or pre-natal influences as appeared to be the case for Malin & Till.1

This study is restricted to evaluating the effect of including both elevation and fluoridation extent as risk-modifying factors rather than evaluating any specific mechanism so no attempt has been made to look at the effect of timing for the different covariates. An initial multiple regression used the 2011 ADHD prevalence and included only the 1992 fluoridation extent and median household income data used by Malin & Till1 and mean state elevation used by Huber et al.2 as covariates.

This initial multiple regression indicated a statistically significant association with median household income (P = 0.009) and mean state elevation (P = 0.002) but there was no significant association with fluoridation extent (P = 0.269) (see Table 2). Inclusion of median household income and mean state elevation together accounted for 40.4% of the variability in the ADHD data. So, simple inclusion of elevation as a risk-modifying factor suggests the relationship of ADHD prevalence with exposure to fluoridated water reported by Malin & Till1 is misleading.

When all the covariates listed in Table 1 were considered in a multiple regression analyses only mean state elevation and per capita personal income in 2009 were statistically significant (see Table 3). The final model was:

ADHD–2011 = 20.2752 −0.00086 × mean state elevation (ft) – 0.00019 personal income 2009 ($), R = −0.680, P = 0.000.

This accounted for 46.2% of the variability in the ADHD data.

Huber et al.2 advanced a mechanistic explanation for the association of ADHD with elevation based on the suggestion that mild hypobaric hypoxia at higher altitudes can increase brain dopamine levels. However, they did suggest caution in interpreting the results as research has indicated a range of possible risk-modifying factors associated with ADHD and suggested the results could also be explained by a previously reported association between solar intensity and ADHD prevalence. Similarly, per capita personal income may be acting as a proxy for some more fundamental social factor in this model.

The loss of the association of ADHD with fluoridation when elevation is included as a covariate suggests fluoridation extent and elevation are not independent variables. In fact, CWF is significantly correlated with elevation (CWF 1992 = 76.55–0.00656*elevation. R = 0.432, P = 0.002) (see Fig. 1).

Figure 1: Relationship between extent of water fluoridation in a state and the mean state elevation.
figure 1

Note: Red line is fitted equation %state fluoridated in 1992 = 77.62-0.00072 × mean state elevation (ft). The equation accounts for 23% of the variability in %state fluoridated data (R2 = 0.227)

CWF was also correlated with mean state elevation in other years considered – 2000, 20002, 2004, 2006, 2008 and 2010 with Pearson correlation coefficients of −0.48 to −0.38 and P values of <0.001 to 0.006.

This illustrates a problem that can occur in ecological studies when important risk-modifying factors are not included in the statistical analyses. It is possible that there is a negative relationship between communities of a size suitable for CWF and elevation so that the data for extent of CWF is acting as a proxy for something more fundamental. Maybe the number and size of cities (and hence extent of fluoridation) decreases with altitude as most major cities probably developed near ports and were therefore likely to be at, or near, sea level. Cohen & Small11 reported a global trend of concentration of larger populations at lower altitudes with 33.5% of the world's population living within 100 vertical meters of sea level while only 15.6% of all inhabited land lies below 100 m elevation. They suggested this arose from the historical tendency to settle in low lying areas in the non-tropical north as this facilitated agriculture, transport and trade.

The lack of consideration of other important risk-modifying factors could be a problem in some other studies relating health problems to fluoridation. Public data for extent of fluoridation is readily available but may not be so available for other important factors. Another example of this problem is shown in a brief unpublished analysis by Perrott12 of the study by Takahashi et al.13 which reported statistically significant relationships of a range of cancers to fluoridation. Takahashi et al.,13 did not consider other possible risk-modifying factors but when multiple regressions included several geographic factors (elevation, longitude and latitude) together with fluoridation the significant associations with fluoridation disappeared.

This example underlines the need for considering all possible risk-modifying factors when undertaking exploratory statistical analysis. In particular, Malin & Till1 included only median household income as a covariate in their statistical analysis. Their introductory comments indicated a preference for a chemical toxicity model and their discussion of the literature was limited to reports claiming a toxic role for fluoride. In contrast Huber et al.2 did not consider fluoridation or other chemical toxicity mechanisms but included a range of possible social and psychological factors as covariates.

There is always the provision in such ecological studies that a statistically significant correlation is not proof of causation – even when a range of possible covariates are included in the analysis. Malin & Till1 acknowledged this, suggesting the need for further work, including consideration of other covariates. However, such scientific qualification is quickly lost when reported relationships are argued as evidence in the political sphere.

Community water fluoridation is a hotly debated political issue and claims of scientific evidence play a prominent role in this debate. Motivated activists will search for, and cite, any evidence they can find of statistically significant relationships and advance these as evidence for harmful effects resulting from CWF. This has been done for hypothyroidism and particularly for IQ deficits.

Most cited studies of harmful effects from fluoride are based on data from areas of endemic fluorosis where drinking water fluoride concentrations are high, with very few studies using data where CWF is used. Anti-fluoridation campaigners have therefore been quick to criticise those studies showing no harmful effect from CWF (for example that of Broadbent et al.)14 and to promote studies suggesting negative effects from CWF, such as that of Malin & Till.1 This has been heavily promoted and cited in recent anti-fluoridation campaigns.

The suggestion by Malin & Till1 of a relationship between CWF and ADHD prevalence is not supported once other important risk-modifying factors, especially mean state elevation, are included in the statistical analysis. The authors may have not been aware at the time of their study of the hypothesis advanced by Huber et al.2 but should have considered a wider number of covariates considering their investigation was exploratory.

CWF reduces the risk of dental caries for both children and adults. After more than seventy years' experience worldwide, systematic reviews (for example Oral Health in America,15 Australian National Health and Medical Research Council Review,16 and the New Zealand Fluoridation Review)17 continue to show it is effective and also fail to find any convincing evidence of harm. Research into water fluoridation will continue, but health professionals and the general public should be assured that the current evidence strongly supports the continuation of this public health initiative.

Conclusions

This critique briefly repeats the statistical analyses of Malin & Till1 and Huber et al.2 using the same data for US states and data for other possible risk-modifying factors available from similar sources. The association of ADHD with extent of fluoridation disappears once elevation is included in the regression analysis. The only covariates showing a statistically significant association were mean state elevation and per capita personal income in 2009. A linear model including mean state elevation and per capita personal income for 2009 accounted for 46% of the ADHD variance. The claim of an association of the extent of community water fluoridation with ADHD prevalence by Malin & Till1 appears unfounded and their conclusion was based on inadequate consideration of risk-modifying factors.