Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Techniques and Methods Validated Methods

Energy intake estimation using a novel wearable sensor and food images in a laboratory (pseudo-free-living) meal setting: quantification and contribution of sources of error

International Journal of Obesity volume 46, pages 2050–2057 (2022)Cite this article

Subjects

Abstract

Objectives

Dietary assessment methods not relying on self-report are needed. The Automatic Ingestion Monitor 2 (AIM-2) combines a wearable camera that captures food images with sensors that detect food intake. We compared energy intake (EI) estimates of meals derived from AIM-2 chewing sensor signals, AIM-2 images, and an internet-based diet diary, with researcher conducted weighed food records (WFR) as the gold standard.

Subjects/Methods

Thirty adults wore the AIM-2 for meals self-selected from a university food court on one day in mixed laboratory and free-living conditions. Daily EI was determined from a sensor regression model, manual image analysis, and a diet diary and compared with that from WFR. A posteriori analysis identified sources of error for image analysis and WFR differences.

Results

Sensor-derived EI from regression modeling (R2 = 0.331) showed the closest agreement with EI from WFR, followed by diet diary estimates. EI from image analysis differed significantly from that by WFR. Bland–Altman analysis showed wide limits of agreement for all three test methods with WFR, with the sensor method overestimating at lower and underestimating at higher EI. Nutritionist error in portion size estimation and irreconcilable differences in portion size between food and nutrient databases used for WFR and image analyses were the greatest contributors to image analysis and WFR differences (44.4% and 44.8% of WFR EI, respectively).

Conclusions

Estimation of daily EI from meals using sensor-derived features offers a promising alternative to overcome limitations of self-report. Image analysis may benefit from computerized analytical procedures to reduce identified sources of error.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of the AIM-2.
Fig. 2: Comparison of methods for assessing daily energy intake (EI).
Fig. 3: Bland–Altman plots showing mean differences and 2 SD limits of agreement.

Similar content being viewed by others

Data availability

Data may be made available upon request to the corresponding authors pending IRB approval.

References

  1. Hall KD. Challenges of human nutrition research. Science. 2020;367:1298–300.

    Article  CAS  Google Scholar 

  2. Shim J-S, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. Epidemiol Health. 2014;36. https://doi.org/10.4178/epih/e2014009.

  3. Free Calorie Counter, Diet & Exercise Journal | MyFitnessPal.com. https://www.myfitnesspal.com/ (accessed 25 Jun 2018).

  4. Lose It! - Weight loss that fits. https://www.loseit.com/ (accessed 22 Nov 2017).

  5. MyNetDiary - Free Calorie Counter and Diet Assistant. https://www.mynetdiary.com/ (accessed 8 Dec 2020).

  6. Cordeiro F, Epstein DA, Thomaz E, Bales E, Jagannathan AK, Abowd GD, et al. Barriers and negative nudges: exploring challenges in food journaling. Proc SIGCHI Conf Hum Factors Comput Syst CHI Conf. 2015;2015:1159–62.

    Article  Google Scholar 

  7. Gill S, Panda S. A smartphone app reveals erratic diurnal eating patterns in humans that can be modulated for health benefits. Cell Metab. 2015;22:789–98.

    Article  CAS  Google Scholar 

  8. Höchsmann C, Martin CK. Review of the validity and feasibility of image-assisted methods for dietary assessment. Int J Obes. 2020;44:2358–71.

    Article  Google Scholar 

  9. Fontana JM, Higgins JA, Schuckers SC, Bellisle F, Pan Z, Melanson EL, et al. Energy intake estimation from counts of chews and swallows. Appetite. 2015;85:14–21.

    Article  Google Scholar 

  10. Bell BM, Alam R, Alshurafa N, Thomaz E, Mondol AS, Haye Kdela, et al. Automatic, wearable-based, in-field eating detection approaches for public health research: a scoping review. Npj Digit Med. 2020;3:1–14.

    Article  Google Scholar 

  11. Hong W, Lee WG. Wearable sensors for continuous oral cavity and dietary monitoring toward personalized healthcare and digital medicine. Analyst. 2020. https://doi.org/10.1039/D0AN01484B.

  12. Doulah A, Mccrory MA, Higgins JA, Sazonov E. A systematic review of technology-driven methodologies for estimation of energy intake. IEEE Access. 2019;7:49653–68.

    Article  Google Scholar 

  13. Doulah A, Farooq M, Yang X, Parton J, McCrory MA, Higgins JA, et al. Meal microstructure characterization from sensor-based food intake detection. Front Nutr 2017;4. https://doi.org/10.3389/fnut.2017.00031.

  14. Amft O, Kusserow M, Troster G. Bite weight prediction from acoustic recognition of chewing. IEEE Trans Biomed Eng. 2009;56:1663–72.

    Article  Google Scholar 

  15. Päßler S, Fischer W-J. Food intake monitoring: automated chew event detection in chewing sounds. IEEE J Biomed Health Inform. 2014;18:278–89.

    Article  Google Scholar 

  16. Alshurafa N, Kalantarian H, Pourhomayoun M, Liu JJ, Sarin S, Shahbazi B, et al. Recognition of nutrition intake using time-frequency decomposition in a wearable necklace using a piezoelectric sensor. IEEE Sens J. 2015;15:3909–16.

    Article  CAS  Google Scholar 

  17. Dong Y, Hoover A, Scisco J, Muth E. A new method for measuring meal intake in humans via automated wrist motion tracking. Appl Psychophysiol Biofeedback. 2012;37:205–15.

    Article  Google Scholar 

  18. Bi Y, Lv M, Song C, Xu W, Guan N, Yi W. AutoDietary: A wearable acoustic sensor system for food intake recognition in daily life. IEEE Sens J. 2016;16:806–16.

    Article  Google Scholar 

  19. Bi S, Wang T, Tobias N, Nordrum J, Wang S, Halvorsen G, et al. Auracle: Detecting eating episodes with an ear-mounted sensor. Proc ACM Interact Mob Wearable Ubiquitous Technol. 2018;2:92:1–92:27.

    Article  Google Scholar 

  20. Amft O, Tröster G. Recognition of dietary activity events using on-body sensors. Artif Intell Med. 2008;42:121–36.

    Article  Google Scholar 

  21. Zhang R, Bernhart S, Amft O. Diet eyeglasses: Recognising food chewing using EMG and smart eyeglasses. In: 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN). 2016, pp 7–12.

  22. Mirtchouk M, Merck C, Kleinberg S. Automated estimation of food type and amount consumed from body-worn audio and motion sensors. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM: New York, NY, USA, 2016, pp 451–62.

  23. Doulah A, Ghosh T, Hossain D, Imtiaz MH, Sazonov E. “Automatic Ingestion Monitor Version 2”—A novel wearable device for automatic food intake detection and passive capture of food images. IEEE J Biomed Health Inform. 2021;25:567–76. https://doi.org/10.1109/JBHI.2020.2995473.

    Article  Google Scholar 

  24. Automated Self-Administered 24-Hour (ASA24®) Dietary Assessment Tool. https://epi.grants.cancer.gov/asa24/ (accessed 22 Nov 2017).

  25. Random permutation of integers - MATLAB randperm. https://www.mathworks.com/help/matlab/ref/randperm.html (accessed 22 Nov 2017).

  26. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. 1996;58:267–88.

    Google Scholar 

  27. Sammut C, Webb GI (eds.). Leave-one-out cross-validation. In: Encyclopedia of Machine Learning. Springer US: Boston, MA, 2010, pp 600–1.

  28. Wang S, Zhou G, Ma Y, Hu L, Chen Z, Chen Y, et al. Eating detection and chews counting through sensing mastication muscle contraction. Smart Health. 2018;9–10:179–91.

    Article  Google Scholar 

  29. FNDDS: USDA ARS. https://www.ars.usda.gov/northeast-area/beltsville-md/beltsville-human-nutrition-research-center/food-surveys-research-group/docs/fndds/ (accessed 22 Nov 2017).

  30. USDA National Nutrient Database for Standard Reference. 2013. www.ars.usda.gov/Services/docs.htm?docid=8964.

  31. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–60.

    Article  CAS  Google Scholar 

  32. Yang X, Doulah A, Farooq M, Parton J, McCrory MA, Higgins JA, et al. Statistical models for meal-level estimation of mass and energy intake using features derived from video observation and a chewing sensor. Sci Rep. 2019;9:45.

    Article  Google Scholar 

  33. Fontana JM, Farooq M, Sazonov E. Automatic Ingestion Monitor: A novel wearable device for monitoring of ingestive behavior. IEEE Trans Biomed Eng. 2014;61:1772–9.

    Article  Google Scholar 

  34. Lorenzoni G, Bottigliengo D, Azzolina D, Gregori D. Food composition impacts theaccuracy of wearable devices when estimating energy intake from energy-dense food. Nutrients. 2019;11. https://doi.org/10.3390/nu11051170.

  35. Nicklas T, Islam NG, Saab R, Schulin R, Liu Y, Butte NF, et al. Validity of a digital diet estimation method for use with preschool children. J Acad Nutr Diet. 2018;118:252–60.

    Article  Google Scholar 

  36. Pan Z, Forjan D, Marden T, Padia J, Ghosh T, Hossain D. et al. Improvement of methodology for manual energy intake estimation from passive capture devices. Front Nutr. 2022;9:877775. https://doi.org/10.3389/fnut.2022.877775.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Funding

Research reported in this publication was supported by the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under Award Number R01DK100796 and by NIH/NCATS Colorado CTSA Grant Number UL1 TR002535. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Author notes

  1. These authors contributed equally: Megan A. McCrory, Edward Sazonov.

Authors and Affiliations

  1. Department of Electrical and Computer Engineering (ECE), The University of Alabama, Tuscaloosa, USA

    Abul Doulah, Tonmoy Ghosh, Delwar Hossain & Edward Sazonov

  2. Department of Electrical and Electronic Engineering, University of Liberal Arts Bangladesh, Dhaka, Bangladesh

    Abul Doulah

  3. Colorado Clinical and Translational Sciences Institute, University of Colorado Anschutz Medical Campus, Aurora, CO, USA

    Tyson Marden

  4. Department of Information Systems, Statistics, and Management Science, Culverhouse College of Commerce and Business Administration, University of Alabama, Tuscaloosa, AL, USA

    Jason M. Parton

  5. Department of Pediatrics, University of Colorado, Anschutz Medical Campus, Denver, CO, USA

    Janine A. Higgins

  6. Department of Health Sciences, Boston University, Boston, MA, USA

    Megan A. McCrory

Authors
  1. Abul Doulah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tonmoy Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Delwar Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tyson Marden
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jason M. Parton
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Janine A. Higgins
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Megan A. McCrory
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Edward Sazonov
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JAH, MAM, and ES conceived and/or designed the work that led to the submission, and agree to be accountable for all aspects of the work ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved; AD, TG, DH, and TM acquired data; AD drafted the manuscript; JAH, MAM, and ES revised the manuscript; all authors played an important role in interpreting the results and approved the final version.

Corresponding authors

Correspondence to Megan A. McCrory or Edward Sazonov.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doulah, A., Ghosh, T., Hossain, D. et al. Energy intake estimation using a novel wearable sensor and food images in a laboratory (pseudo-free-living) meal setting: quantification and contribution of sources of error. Int J Obes 46, 2050–2057 (2022). https://doi.org/10.1038/s41366-022-01225-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41366-022-01225-w

This article is cited by

Search

Advanced search

Quick links