Testing the carbohydrate-insulin model of obesity in a 5-month feeding study: the perils of post-hoc participant exclusions

A large feeding study reported that total energy expenditure (TEE) was greater on a low- versus high-carbohydrate diet, supporting the carbohydrate-insulin model of obesity. Recently, the validity of this finding was challenged in a post-hoc analysis excluding participants with putative non-adherence to the study diets. Here, we show why that analysis, based on a post-randomization variable linked to the outcome, introduced severe confounding bias. With control for confounding, the diet effect on TEE remained strong in a reanalysis. Together with sensitivity analyses demonstrating robustness to plausible levels of non-adherence, these data provide experimental support for a potentially novel metabolic effect of macronutrients that might inform the design of more effective obesity treatment.


Introduction
Are all calories metabolically alike to the body? According the carbohydrate-insulin model of obesity (CIM) [1,2], the high insulin-to-glucagon ratio on a high-glycemic load diet shifts substrate partitioning from lean tissue to adipose, lowers the concentration of metabolic fuels in the blood in the late postprandial period, increases hunger and lowers energy expenditurebiological responses that would tend to promote weight gain. Thus, the CIM offers a physiological explanation for why average BMI in many countries increased in the late 20 th century as public health guidelines recommended replacement of dietary fat with carbohydrate, and consumption of high-glycemic load foods (chiefly processed grains, potato products and added sugars) increased substantially.
One prediction of the CIM is that, during weight-loss maintenance, total energy expenditure (TEE) would increase with carbohydrate restriction, an effect estimated to be approximately +50 kcal/d for every 10% decrease in dietary carbohydrate as an absolute proportion of total energy intake [2]. Although a recent meta-analysis claimed to find no evidence for this metabolic effect in macronutrient-controlled trials [3], the median duration of included studies was <7 days, an insufficient timeframe to allow for the well-described transient changes that occur over 2 to 3 weeks with reduction in carbohydrate intake [2].
In 2018, our group reported the results of a large feeding study comparing low-(20%), moderate-(40%), and high-(60%) carbohydrate diets throughout 5 months weight-loss maintenance (i.e., with adjustment of dietary energy to prevent weight change). We found that TEE measured by doubly-labeled water (DLW) was approximately 250 kcal/d greater on the low-vs high-carbohydrate diet [4]. If this effect were reproducible and durable, it might translate into substantial weight loss under natural conditions (i.e., without experimental control of dietary intake).
Soon after publication of this study, one group reanalyzed the publicly available data by eliminating participants according to "Unaccounted Energy" (UE), a calculated variable intended to reflect non-adherence to the test diets [5]. They compared individual levels of energy intake with TEE, adjusting for change in body weight, and sequentially excluded those with high UE as shown in Fig. 1a. In this analysis, the dietary effect on TEE seems to decrease by nearly half, comparing the full cohort represented by the rightmost data point with the left-most point from which 50% of participants had been eliminated. This relationship was interpreted as demonstrating that dietary non-adherence invalidated study outcomes due to the dependency of DLW methodology on assumptions about respiratory quotient (RQ).
In a recent report [6], we argued this exclusion analysis, based on post-randomization factors, risks introducing severe bias because of the implicit assumption that UE is causally related to the outcome. In fact, this apparent relationship could be influenced by any baseline or postrandomization factor associated with measurement of both UE and TEE. Once data are excluded in this way, the protection from randomization is lost, and the analyses become subject to the major methodological limitations of observational research, most importantly confounding. Here, we employed conventional epidemiological methods to examine the validity of the reanalysis.

Methods and results
Statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC). Since individual-level dietary data, a secondary outcome in our original study, were preliminary, we first obtained complete dietary intake [7]. We calculated UE as the absolute value of the difference between energy intake and energy expenditure, correcting for change in body fat mass by isotope dilution (see [7] for details on determination of, and adjustment for, body composition). Using the median value of 438.1 kcal/d, we dichotomized the cohort between those with the highest vs lowest UE, the former comprising those excluded in Panel A.
As shown in Table 1, the excluded group had more males; were younger, heavier and taller; had lost less weight during the weight-loss run-in; and most notably, had higher baseline TEE. They also had a numerically higher baseline insulin-30 (a marker of insulin secretion, an effect modifier) [4]. Each of these covariates would likely be associated with a greater magnitude of TEE outcome in absolute terms, independent of UE.
We repeated the exclusion analysis using repeated measures ANOVA to test the effect of diet on TEE while controlling for the above-mentioned baseline covariates, insulin-30, and study cohort (the latter to control for temporal effects over the 3-year study, consistent with our original analysis plan) [4]. Change in TEE was expressed as kcal/day/kg normalized to the average post-weight loss weight (82 kg). As shown in Fig. 1b, TEE diet effect for the low-vs high-carbohydrate diet declined with sequential exclusions markedly less with covariate adjustment than without adjustment (attenuation of 9% vs 42%). Moreover, there was no difference in TEE diet effect among those in the lowest vs highest category of UE (p = 0.75).

Discussion
By eliminating 50% of the sample with highest UE, only 9% of the originally observed diet effect on TEE might be attributable to non-adherence. Most likely, the amount so attributable would be even less, as the reanalysis is still subject to confounding by post-randomization factors beyond our control. Specifically, individuals who have a bigger than average diet-induced increase in TEE from measurement error (and consequently a larger apparent UE) would be excluded to a greater extent than those with a smaller than average increase from measurement error in the opposite direction. In this fashion, one tail of a distribution curve would be selectively depleted, producing unbalanced (systematic) error with bias toward the null hypothesis. Moreover, the very notion of UE is potentially misleading, because it comprises multiple components of energy balance (intake, expenditure, and body composition) that each have measurement imprecision. An individual might have high UE arising not only from dietary non-adherence, but also cumulative random measurement error. Even in the optimal experimental environment afforded by a metabolic ward, most participants in a 2-week feeding study had UE ≥ 250 kcal/d [8].
The physiological mechanisms relating a lowcarbohydrate diet to higher energy expenditure have been considered elsewhere [1,2,4,7] and may involve lower insulin and ghrelin action in adipose tissue, higher glucagon action in non-adipose sites, increased leptin sensitivity in muscle, and multiple hormonal and metabolic signals acting in the brain. Conversely, on a high-carbohydrate diet, thyroid hormone level (though not necessarily hormone sensitivity) and sympathetic nervous system activity are higher. Although more research will be needed to clarify mechanisms, the effects we observed are independent of body composition [7], and likely to involve a major contribution from non-resting energy expenditure, consistent with current understanding of adaptive thermogenesis during weight-loss maintenance [9].
Sensitivity analyses also provide confidence in the validity of the study's main findings. The higher TEE on the low-vs high-carbohydrate diet remained statistically significant up to 50% non-adherence, using conservative assumptions for RQ [6]. Furthermore, the diet effect on energy requirement was not attenuated when participants with energy intake to expenditure ratio higher and lower than average were eliminated (i.e., those at both tails of the distribution) [7]. Finally, new data on energy requirements for weight-loss maintenance show effects commensurate with TEE (about 200 to 300 kcal greater on the low-vs high-carbohydrate diet) [7]. Thus, our study provides support for the CIM and for a potentially novel effect of dietary macronutrients on metabolism. In view of the methodological limitations of all feeding studies, additional research into this macronutrient effect is warranted. More broadly, our current reanalysis highlights the well-described perils of excluding data from an RCT based on post-randomization variables [10]. Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons. org/licenses/by/4.0/.