A large feeding study reported that total energy expenditure (TEE) was greater on a low- versus high-carbohydrate diet, supporting the carbohydrate-insulin model of obesity. Recently, the validity of this finding was challenged in a post-hoc analysis excluding participants with putative non-adherence to the study diets. Here, we show why that analysis, based on a post-randomization variable linked to the outcome, introduced severe confounding bias. With control for confounding, the diet effect on TEE remained strong in a reanalysis. Together with sensitivity analyses demonstrating robustness to plausible levels of non-adherence, these data provide experimental support for a potentially novel metabolic effect of macronutrients that might inform the design of more effective obesity treatment.
Are all calories metabolically alike to the body? According the carbohydrate-insulin model of obesity (CIM) [1, 2], the high insulin-to-glucagon ratio on a high-glycemic load diet shifts substrate partitioning from lean tissue to adipose, lowers the concentration of metabolic fuels in the blood in the late postprandial period, increases hunger and lowers energy expenditure – biological responses that would tend to promote weight gain. Thus, the CIM offers a physiological explanation for why average BMI in many countries increased in the late 20th century as public health guidelines recommended replacement of dietary fat with carbohydrate, and consumption of high-glycemic load foods (chiefly processed grains, potato products and added sugars) increased substantially.
One prediction of the CIM is that, during weight-loss maintenance, total energy expenditure (TEE) would increase with carbohydrate restriction, an effect estimated to be approximately +50 kcal/d for every 10% decrease in dietary carbohydrate as an absolute proportion of total energy intake . Although a recent meta-analysis claimed to find no evidence for this metabolic effect in macronutrient-controlled trials , the median duration of included studies was <7 days, an insufficient timeframe to allow for the well-described transient changes that occur over 2 to 3 weeks with reduction in carbohydrate intake .
In 2018, our group reported the results of a large feeding study comparing low- (20%), moderate- (40%), and high- (60%) carbohydrate diets throughout 5 months weight-loss maintenance (i.e., with adjustment of dietary energy to prevent weight change). We found that TEE measured by doubly-labeled water (DLW) was approximately 250 kcal/d greater on the low- vs high-carbohydrate diet . If this effect were reproducible and durable, it might translate into substantial weight loss under natural conditions (i.e., without experimental control of dietary intake).
Soon after publication of this study, one group reanalyzed the publicly available data by eliminating participants according to “Unaccounted Energy” (UE), a calculated variable intended to reflect non-adherence to the test diets . They compared individual levels of energy intake with TEE, adjusting for change in body weight, and sequentially excluded those with high UE as shown in Fig. 1a. In this analysis, the dietary effect on TEE seems to decrease by nearly half, comparing the full cohort represented by the right-most data point with the left-most point from which 50% of participants had been eliminated. This relationship was interpreted as demonstrating that dietary non-adherence invalidated study outcomes due to the dependency of DLW methodology on assumptions about respiratory quotient (RQ).
In a recent report , we argued this exclusion analysis, based on post-randomization factors, risks introducing severe bias because of the implicit assumption that UE is causally related to the outcome. In fact, this apparent relationship could be influenced by any baseline or post-randomization factor associated with measurement of both UE and TEE. Once data are excluded in this way, the protection from randomization is lost, and the analyses become subject to the major methodological limitations of observational research, most importantly confounding. Here, we employed conventional epidemiological methods to examine the validity of the reanalysis.
Methods and results
Statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC). Since individual-level dietary data, a secondary outcome in our original study, were preliminary, we first obtained complete dietary intake . We calculated UE as the absolute value of the difference between energy intake and energy expenditure, correcting for change in body fat mass by isotope dilution (see  for details on determination of, and adjustment for, body composition). Using the median value of 438.1 kcal/d, we dichotomized the cohort between those with the highest vs lowest UE, the former comprising those excluded in Panel A.
As shown in Table 1, the excluded group had more males; were younger, heavier and taller; had lost less weight during the weight-loss run-in; and most notably, had higher baseline TEE. They also had a numerically higher baseline insulin-30 (a marker of insulin secretion, an effect modifier) . Each of these covariates would likely be associated with a greater magnitude of TEE outcome in absolute terms, independent of UE.
We repeated the exclusion analysis using repeated measures ANOVA to test the effect of diet on TEE while controlling for the above-mentioned baseline covariates, insulin-30, and study cohort (the latter to control for temporal effects over the 3-year study, consistent with our original analysis plan) . Change in TEE was expressed as kcal/day/kg normalized to the average post-weight loss weight (82 kg). As shown in Fig. 1b, TEE diet effect for the low- vs high-carbohydrate diet declined with sequential exclusions markedly less with covariate adjustment than without adjustment (attenuation of 9% vs 42%). Moreover, there was no difference in TEE diet effect among those in the lowest vs highest category of UE (p = 0.75).
By eliminating 50% of the sample with highest UE, only 9% of the originally observed diet effect on TEE might be attributable to non-adherence. Most likely, the amount so attributable would be even less, as the reanalysis is still subject to confounding by post-randomization factors beyond our control. Specifically, individuals who have a bigger than average diet-induced increase in TEE from measurement error (and consequently a larger apparent UE) would be excluded to a greater extent than those with a smaller than average increase from measurement error in the opposite direction. In this fashion, one tail of a distribution curve would be selectively depleted, producing unbalanced (systematic) error with bias toward the null hypothesis. Moreover, the very notion of UE is potentially misleading, because it comprises multiple components of energy balance (intake, expenditure, and body composition) that each have measurement imprecision. An individual might have high UE arising not only from dietary non-adherence, but also cumulative random measurement error. Even in the optimal experimental environment afforded by a metabolic ward, most participants in a 2-week feeding study had UE ≥ 250 kcal/d .
The physiological mechanisms relating a low-carbohydrate diet to higher energy expenditure have been considered elsewhere [1, 2, 4, 7] and may involve lower insulin and ghrelin action in adipose tissue, higher glucagon action in non-adipose sites, increased leptin sensitivity in muscle, and multiple hormonal and metabolic signals acting in the brain. Conversely, on a high-carbohydrate diet, thyroid hormone level (though not necessarily hormone sensitivity) and sympathetic nervous system activity are higher. Although more research will be needed to clarify mechanisms, the effects we observed are independent of body composition , and likely to involve a major contribution from non-resting energy expenditure, consistent with current understanding of adaptive thermogenesis during weight-loss maintenance .
Sensitivity analyses also provide confidence in the validity of the study’s main findings. The higher TEE on the low- vs high-carbohydrate diet remained statistically significant up to 50% non-adherence, using conservative assumptions for RQ . Furthermore, the diet effect on energy requirement was not attenuated when participants with energy intake to expenditure ratio higher and lower than average were eliminated (i.e., those at both tails of the distribution) . Finally, new data on energy requirements for weight-loss maintenance show effects commensurate with TEE (about 200 to 300 kcal greater on the low- vs high-carbohydrate diet) . Thus, our study provides support for the CIM and for a potentially novel effect of dietary macronutrients on metabolism. In view of the methodological limitations of all feeding studies, additional research into this macronutrient effect is warranted. More broadly, our current reanalysis highlights the well-described perils of excluding data from an RCT based on post-randomization variables .
Ludwig DS. The glycemic index: physiological mechanisms relating to obesity, diabetes, and cardiovascular disease. JAMA. 2002;287:2414–23.
Ludwig DS, Ebbeling CB. The carbohydrate-insulin model of obesity: beyond “calories in, calories out”. JAMA Int Med. 2018;178:1098–103.
Hall KD, Guo J. Obesity energetics: body weight regulation and the effects of diet composition. Gastroenterology. 2017;152:1718–27.e3.
Ebbeling CB, Feldman HA, Klein GL, Wong JMW, Bielak L, Steltz SK, et al. Effects of a low carbohydrate diet on energy expenditure during weight loss maintenance: randomized trial. BMJ. 2018;363:k4583. https://doi.org/10.1136/bmj.k4583.
Hall KD, Guo J, Speakman JR. Do low-carbohydrate diets increase energy expenditure? Int J Obes (Lond). 2019;43:2350–4.
Ludwig DS, Lakin PR, Wong WW, Ebbeling CB. Scientific discourse in the era of open science: a response to Hall et al. regarding the Carbohydrate-Insulin Model. Int J Obes (Lond). 2019;43:2355–60.
Ebbeling CB, Bielak L, Lakin PR, Klein GL, Wong JMW, Luoto PK, et al. Energy requirement is higher during weight-loss maintenance in adults consuming a low- compared with high-carbohydrate diet. J Nutr. 2020, in press.
Hall KD, Ayuketah A, Brychta R, Cai H, Cassimatis T, Chen KY, et al. Ultra-processed diets cause excess calorie intake and weight gain: an inpatient randomized controlled trial of ad libitum food intake. Cell Metab. 2019;30:67–77.
Müller MJ, Enderle J, Bosy-Westphal A. Changes in energy expenditure with weight gain and weight loss in humans. Curr Obes Rep. 2016;5:413–23.
Piantadosi S. Random error and bias (Chpt 6) In: Clinical trials: a methodological perspective, 3rd ed. Hoboken, NJ: John Wiley & Sons, Inc.; 2017.
The trial considered here was funded by Nutrition Science Initiative (made possible by gifts from Arnold Ventures and the Robert Lloyd Corkin Charitable Foundation), New Balance Foundation, Many Voices Foundation, and Blue Cross Blue Shield. DSL was supported by a mid-career mentoring award from the National Institute of Diabetes and Digestive and Kidney Diseases (K24DK082730). The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the study sponsors.
The protocol and data set for the original trial findings and the new data on energy intake are available at Open Science Framework (https://osf.io/rvbuy/).
Conflict of interest
DSL and CBE have conducted research studies examining the carbohydrate-insulin model of obesity funded by the National Institutes of Health and philanthropic organizations unaffiliated with the food industry; DSL received royalties for books on obesity and nutrition that recommend a low-glycemic load diet. No other author has relevant disclosures.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Ludwig, D.S., Greco, K.F., Ma, C. et al. Testing the carbohydrate-insulin model of obesity in a 5-month feeding study: the perils of post-hoc participant exclusions. Eur J Clin Nutr (2020). https://doi.org/10.1038/s41430-020-0658-8