Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Techniques and Methods

Review of the validity and feasibility of image-assisted methods for dietary assessment

International Journal of Obesity volume 44, pages 2358–2371 (2020)Cite this article

Subjects

Abstract

Accurately quantifying dietary intake is essential to understanding the effect of diet on health and evaluating the efficacy of dietary interventions. Self-report methods (e.g., food records) are frequently utilized despite evident inaccuracy of these methods at assessing energy and nutrient intake. Methods that assess food intake via images of foods have overcome many of the limitations of traditional self-report. In cafeteria settings, digital photography has proven to be unobtrusive and accurate and is the method of choice for assessing food provision, plate waste, and food intake. In free-living conditions, image capture of food selection and plate waste via the user’s smartphone, is promising and can produce accurate energy intake estimates, though accuracy is not guaranteed. These methods foster (near) real-time transfer of data and eliminate the need for portion size estimation by the user since the food images are analyzed by trained raters. A limitation that remains, similar to self-report methods where participants must truthfully record all consumed foods, is intentional and/or unintentional underreporting of foods due to social desirability or forgetfulness. Methods that rely on passive image capture via wearable cameras are promising and aim to reduce user burden; however, only pilot data with limited validity are currently available and these methods remain obtrusive and cumbersome. To reduce analysis-related staff burden and to allow real-time feedback to the user, recent approaches have aimed to automate the analysis of food images. The technology to support automatic food recognition and portion size estimation is, however, still in its infancy and fully automated food intake assessment with acceptable precision not yet a reality. This review further evaluates the benefits and challenges of current image-assisted methods of food intake assessment and concludes that less burdensome methods are less accurate and that no current method is adequate in all settings.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of different dietary assessment methods concerning accuracy, unobtrusiveness, analysis time, participant burden, staff burden, and food waste detection.

Similar content being viewed by others

References

  1. Subar AF, Freedman LS, Tooze JA, Kirkpatrick SI, Boushey C, Neuhouser ML, et al. Addressing current criticism regarding the value of self-report dietary data. J Nutr. 2015;145:2639–45.

    CAS  PubMed  PubMed Central  Google Scholar 

  2. Walter W. Nutritional epidemiology. 3rd ed. Oxford, New York: Oxford University Press; 2012.

    Google Scholar 

  3. Lytle LA, Nicastro HL, Roberts SB, Evans M, Jakicic JM, Laposky AD, et al. Accumulating data to optimally predict obesity treatment (ADOPT) core measures: behavioral domain. Obes Silver Spring Md. 2018;26(Suppl 2):S16–24.

    Google Scholar 

  4. Johnson RK. Dietary intake—how do we measure what people are really eating? Obes Res. 2002;10:63S–8S.

    PubMed  Google Scholar 

  5. Archer E, Pavela G, Lavie CJ. The inadmissibility of ‘What We Eat In America’ (WWEIA) and NHANES dietary data in nutrition & obesity research and the scientific formulation of national dietary guidelines. Mayo Clin Proc. 2015;90:911–26.

    PubMed  PubMed Central  Google Scholar 

  6. Dhurandhar NV, Schoeller D, Brown AW, Heymsfield SB, Thomas D, Sørensen TIA, et al. Energy balance measurement: when something is not better than nothing. Int J Obes. 2015;39:1109–13.

    CAS  Google Scholar 

  7. Basiotis PP, Welsh SO, Cronin FJ, Kelsay JL, Mertz W. Number of days of food intake records required to estimate individual and group nutrient intakes with defined confidence. J Nutr. 1987;117:1638–41.

    CAS  PubMed  Google Scholar 

  8. Macdiarmid J, Blundell J. Assessing dietary intake: who, what and why of under-reporting. Nutr Res Rev. 1998;11:231–53.

    CAS  PubMed  Google Scholar 

  9. Rebro SM, Patterson RE, Kristal AR, Cheney CL. The effect of keeping food records on eating patterns. J Am Diet Assoc. 1998;98:1163–5.

    CAS  PubMed  Google Scholar 

  10. Goris AH, Westerterp-Plantenga MS, Westerterp KR. Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake. Am J Clin Nutr. 2000;71:130–4.

    CAS  PubMed  Google Scholar 

  11. Stone AA, Shiffman S. Ecological momentary assessment (Ema) in behavioral medicine. Ann Behav Med. 1994;16:199–202.

    Google Scholar 

  12. Martin CK, Correa JB, Han H, Allen HR, Rood J, Champagne CM, et al. Validity of the remote food photography method (RFPM) for estimating energy and nutrient intake in near real-time. Obesity (Silver Spring). 2012;20:891–9.

    CAS  Google Scholar 

  13. Williamson DA, Allen HR, Martin PD, Alfonso AJ, Gerald B, Hunt A. Comparison of digital photography to weighed and visual estimation of portion sizes. J Am Diet Assoc. 2003;103:1139–45.

    PubMed  Google Scholar 

  14. Williamson DA, Allen HR, Martin PD, Alfonso A, Gerald B, Hunt A. Digital photography: a new method for estimating food intake in cafeteria settings. Eat Weight Disord. 2004;9:24–8.

    CAS  PubMed  Google Scholar 

  15. Williamson DA, Martin PD, Allen HR, Most MM, Alfonso AJ, Thomas V, et al. Changes in food intake and body weight associated with basic combat training. Mil Med. 2002;167:248–53.

    PubMed  Google Scholar 

  16. Martin CK, Newton RL, Anton SD, Allen HR, Alfonso A, Han H, et al. Measurement of children’s food intake with digital photography and the effects of second servings upon food intake. Eat Behav. 2007;8:148–56.

    PubMed  Google Scholar 

  17. Williamson DA, Champagne CM, Harsha D, Han H, Martin CK, Newton R, et al. Louisiana (LA) health: design and methods for a childhood obesity prevention program in rural schools. Contemp Clin Trials. 2008;29:783–95.

    PubMed  PubMed Central  Google Scholar 

  18. Nicklas TA, O’Neil CE, Stuff J, Goodell LS, Liu Y, Martin CK. Validity and feasibility of a digital diet estimation method for use with preschool children: a pilot study. J Nutr Educ Behav. 2012;44:618–23.

    PubMed  PubMed Central  Google Scholar 

  19. Nicklas TA, O’Neil CE, Stuff JE, Hughes SO, Liu Y. Characterizing dinner meals served and consumed by low-income preschool children. Child Obes Print. 2012;8:561–71.

    Google Scholar 

  20. Nicklas TA, Liu Y, Stuff JE, Fisher JO, Mendoza JA, O’Neil CE. Characterizing lunch meals served and consumed by preschool children in head start. Public Health Nutr. 2013;16:2169–77.

    PubMed  PubMed Central  Google Scholar 

  21. Washburn RA, Honas JJ, Ptomey LT, Mayo MS, Lee J, Sullivan DK, et al. Energy and macronutrient intake in the Midwest Exercise Trial-2 (MET-2). Med Sci Sports Exerc. 2015;47:1941–9.

    CAS  PubMed  PubMed Central  Google Scholar 

  22. Hawkins KR, Burton JH, Apolzan JW, Thomson JL, Williamson DA, Martin CK. Efficacy of a school-based obesity prevention intervention at reducing added sugar and sodium in children’s school lunches: the LA Health randomized controlled trial. Int J Obes. 2018;42:1845–52.

    Google Scholar 

  23. Marcano-Olivier M, Erjavec M, Horne PJ, Viktor S, Pearson R. Measuring lunchtime consumption in school cafeterias: a validation study of the use of digital photography. Public Health Nutr. 2019;22:1745–54.

    PubMed  Google Scholar 

  24. U.S. Department of Agriculture, Agricultural Research Service. USDA Food and Nutrient Database for Dietary Studies 2015-2016. Beltsville, MD: U.S. Department of Agriculture, Agricultural Research Service; 2018.

    Google Scholar 

  25. Martin CK, Han H, Coulon SM, Allen HR, Champagne CM, Anton SD. A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method. Br J Nutr. 2009;101:446–56.

    CAS  PubMed  Google Scholar 

  26. U.S. Environmental Protection Agency, U.S. Food and Drug Administration, U.S. Department of Agriculture. Formal agreement between EPA, USDA, and FDA relative to cooperation and coordination on food loss and waste. Washington, DC: US Food Drug Administration; 2020.

  27. Martin CK, Thomson JL, LeBlanc MM, Stewart TM, Newton RL, Han H, et al. Children in school cafeterias select foods containing more saturated fat and energy than the Institute of Medicine recommendations. J Nutr. 2010;140:1653–60.

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Ptomey LT, Willis EA, Honas JJ, Mayo MS, Washburn RA, Herrmann SD, et al. Validity of energy intake estimated by digital photography + recall in overweight and obese young adults. J Acad Nutr Diet. 2015;115:1392–9.

    PubMed  PubMed Central  Google Scholar 

  29. Martin CK, Nicklas T, Gunturk B, Correa JB, Allen HR, Champagne C. Measuring food intake with digital photography. J Hum Nutr Diet Off J Br Diet Assoc. 2014;27(Suppl 1):72–81.

    Google Scholar 

  30. Nicklas T, Islam NG, Saab R, Schulin R, Liu Y, Butte NF, et al. Validity of a digital diet estimation method for use with preschool children. J Acad Nutr Diet. 2018;118:252–60.

    PubMed  Google Scholar 

  31. Nicklas T, Saab R, Islam NG, Wong W, Butte N, Schulin R, et al. Validity of the remote food photography method against doubly labeled water among minority preschoolers. Obesity (Silver Spring). 2017;25:1633–8.

    Google Scholar 

  32. Altazan AD, Gilmore LA, Burton JH, Ragusa SA, Apolzan JW, Martin CK, et al. Development and application of the remote food photography method to measure food intake in exclusively milk fed infants: a laboratory-based study. PLoS ONE. 2016;11:e0163833.

    PubMed  PubMed Central  Google Scholar 

  33. Duhé AF, Gilmore LA, Burton JH, Martin CK, Redman LM. The remote food photography method accurately estimates dry powdered foods-the source of calories for many infants. J Acad Nutr Diet. 2016;116:1172–7.

    PubMed  PubMed Central  Google Scholar 

  34. Bekelman TA, Bellows LL, McCloskey ML, Martin CK, Johnson SL. Assessing dinner meals offered at home among preschoolers from low-income families with the remote food photography method. Pediatr Obes. 2019;14:e12558.

    PubMed  PubMed Central  Google Scholar 

  35. McCloskey ML, Johnson SL, Bekelman TA, Martin CK, Bellows LL. Beyond nutrient intake: use of digital food photography methodology to examine family dinnertime. J Nutr Educ Behav. 2019;51:547–55.e1.

    PubMed  PubMed Central  Google Scholar 

  36. Buchowski MS. Doubly labeled water is a validated and verified reference standard in nutrition research. J Nutr. 2014;144:573–4.

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Roe BE, Apolzan JW, Qi D, Allen HR, Martin CK. Plate waste of adults in the United States measured in free-living conditions. PLoS ONE. 2018;13:e0191813.

    PubMed  PubMed Central  Google Scholar 

  38. Burrows TL, Martin RJ, Collins CE. A systematic review of the validity of dietary assessment methods in children when compared with the method of doubly labeled water. J Am Diet Assoc. 2010;110:1501–10.

    PubMed  Google Scholar 

  39. Most J, Vallo PM, Altazan AD, Gilmore LA, Sutton EF, Cain LE, et al. Food photography is not an accurate measure of energy intake in obese, pregnant women. J Nutr. 2018;148:658–63.

    PubMed  PubMed Central  Google Scholar 

  40. Rose MH, Streisand R, Aronow L, Tully C, Martin CK, Mackey E. Preliminary feasibility and acceptability of the remote food photography method for assessing nutrition in young children with type 1 diabetes. Clin Pract Pediatr Psychol. 2018;6:270–7.

    PubMed  PubMed Central  Google Scholar 

  41. Martin CK, Kaya S, Gunturk BK. Quantification of food intake using food image analysis. Annu Int Conf IEEE Eng Med Biol Soc. 2009;2009:6869–72.

    PubMed  Google Scholar 

  42. Dibiano R, Gunturk BK, Martin CK. Food image analysis for measuring food intake in free living conditions. Med. Imaging 2013 Image Proces. 2013;8669:86693N.

    Google Scholar 

  43. Casperson SL, Sieling J, Moon J, Johnson L, Roemmich JN, Whigham L. A mobile phone food record app to digitally capture dietary intake for adolescents in a free-living environment: usability study. JMIR MHealth UHealth. 2015;3:e30.

    PubMed  PubMed Central  Google Scholar 

  44. Rollo ME, Ash S, Lyons-Wall P, Russell AW. Evaluation of a mobile phone image-based dietary assessment method in adults with type 2 diabetes. Nutrients. 2015;7:4897–910.

    PubMed  PubMed Central  Google Scholar 

  45. Gemming L, Rush E, Maddison R, Doherty A, Gant N, Utter J, et al. Wearable cameras can reduce dietary under-reporting: doubly labelled water validation of a camera-assisted 24 h recall. Br J Nutr. 2015;113:284–91.

    CAS  PubMed  Google Scholar 

  46. Hodges S, Williams L, Berry E, Izadi S, Srinivasan J, Butler A, et al. SenseCam: a retrospective memory aid. Proceedings of the 8th International Conference of Ubiquitous Computing (UbiComp 2006), Springer-Verlag; 2006. p. 177–93.

  47. Pettitt C, Liu J, Kwasnicki RM, Yang G-Z, Preston T, Frost G. A pilot study to determine whether using a lightweight, wearable micro-camera improves dietary assessment accuracy and offers information on macronutrients and eating rate. Br J Nutr. 2016;115:160–7.

    CAS  PubMed  Google Scholar 

  48. Ahmad Z, Khanna N, Kerr DA, Boushey CJ, Delp EJ. A mobile phone user interface for image-based dietary assessment. Proc SPIE Int Soc Opt Eng. 2014;9030:9030.

    Google Scholar 

  49. Ahmad Z, Kerr DA, Bosch M, Boushey CJ, Delp EJ, Khanna N, et al. A mobile food record for integrated dietary assessment. MADiMa16. 2016;2016:53–62.

    Google Scholar 

  50. Fang S, Liu C, Zhu F, Delp EJ, Boushey CJ. Single-view food portion estimation based on geometric models. 2015 IEEE International Symposium on Multimedia (ISM). Miami, FL: IEEE; 2015, p. 385–90.

  51. Lee CD, Chae J, Schap TE, Kerr DA, Delp EJ, Ebert DS, et al. Comparison of known food weights with image-based portion-size automated estimation and adolescents’ self-reported portion size. J Diabetes Sci Technol. 2012;6:428–34.

    PubMed  PubMed Central  Google Scholar 

  52. Boushey CJ, Spoden M, Delp EJ, Zhu F, Bosch M, Ahmad Z. et al. Reported energy intake accuracy compared to doubly labeled water and usability of the mobile food record among community dwelling adults. Nutrients. 2017;9:312.

    PubMed Central  Google Scholar 

  53. Aflague TF, Boushey CJ, Guerrero RTL, Ahmad Z, Kerr DA, Delp EJ. Feasibility and use of the mobile food record for capturing eating occasions among children ages 3-10 years in Guam. Nutrients. 2015;7:4403–15.

    PubMed  PubMed Central  Google Scholar 

  54. Boushey CJ, Harray AJ, Kerr DA, Schap TE, Paterson S, Aflague T, et al. How willing are adolescents to record their dietary intake? The mobile food record. JMIR MHealth UHealth. 2015;3:e47.

    PubMed  PubMed Central  Google Scholar 

  55. Panizza CE, Boushey CJ, Delp EJ, Kerr DA, Lim E, Gandhi K. et al. Characterizing early adolescent plate waste using the mobile food record. Nutrients. 2017;9:93.

    PubMed Central  Google Scholar 

  56. Shoneye CL, Dhaliwal SS, Pollard CM, Boushey CJ, Delp EJ, Harray AJ. et al. Image-based dietary assessment and tailored feedback using mobile technology: mediating behavior change in young adults. Nutrients. 2019;11:435.

    PubMed Central  Google Scholar 

  57. Fang S, Shao Z, Kerr DA, Boushey CJ, Zhu F. An end-to-end image-based automatic food energy estimation technique based on learned energy distribution images: protocol and methodology. Nutrients. 2019;11:877.

    PubMed Central  Google Scholar 

  58. Rhyner D, Loher H, Dehais J, Anthimopoulos M, Shevchik S, Botwey RH. et al. Carbohydrate estimation by a mobile phone-based system versus self-estimations of individuals with type 1 diabetes mellitus: a comparative study. J Med Internet Res. 2016;18:e101.

    PubMed  PubMed Central  Google Scholar 

  59. Kawano Y, Yanai K. FoodCam: a real-time food recognition system on a smartphone. Multimed Tools Appl. 2015;74:5263–87.

    Google Scholar 

  60. Zhang W, Yu Q, Siddiquie B, Divakaran A, Sawhney H. Snap-n-Eat. J Diabetes Sci Technol. 2015;9:525–33.

    PubMed  PubMed Central  Google Scholar 

  61. Jia W, Chen H-C, Yue Y, Li Z, Fernstrom J, Bai Y, et al. Accuracy of food portion size estimation from digital pictures acquired by a chest-worn camera. Public Health Nutr. 2014;17:1671–81.

    PubMed  Google Scholar 

  62. Cauvain SP, Young LS. Technology of breadmaking. New York, NY: Springer Science & Business Media; 2013.

  63. Holst A. Number of smartphone users worldwide 2014-2020. Statista. 2019. https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/. Accessed 26 Dec 2019.

  64. International Data Corporation. Always connected—how smartphones and social keep us engaged. Framingham, MA: International Data Corporation; 2013.

    Google Scholar 

  65. Howes E, Boushey CJ, Kerr DA, Tomayko EJ, Cluskey M. Image-based dietary assessment ability of dietetics students and interns. Nutrients. 2017;9:114.

    PubMed Central  Google Scholar 

  66. Gibson RS, Charrondiere UR, Bell W. Measurement errors in dietary assessment using self-reported 24-hour recalls in low-income countries and strategies for their prevention. Adv Nutr. 2017;8:980–91.

    PubMed  PubMed Central  Google Scholar 

  67. Maurer J, Taren DL, Teixeira PJ, Thomson CA, Lohman TG, Going SB, et al. The psychosocial and behavioral characteristics related to energy misreporting. Nutr Rev. 2006;64:53–66.

    PubMed  Google Scholar 

  68. Subar AF, Crafts J, Zimmerman TP, Wilson M, Mittl B, Islam NG, et al. Assessment of the accuracy of portion size reports using computer-based food photographs aids in the development of an automated self-administered 24-hour recall. J Am Diet Assoc. 2010;110:55–64.

    PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported by National Institutes of Health grants T32 DK064584 (NIDDK National Research Service Award), P30 DK072476 (Nutrition Obesity Research Center), and U54 GM104940 (Louisiana Clinical and Translational Science Center).

Author information

Authors and Affiliations

  1. Pennington Biomedical Research Center, Baton Rouge, LA, USA

    Christoph Höchsmann & Corby K. Martin

Authors
  1. Christoph Höchsmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Corby K. Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Corby K. Martin.

Ethics declarations

Conflict of interest

The intellectual property surrounding the Remote Food Photography Method© and SmartIntake® application are owned by Louisiana State University/Pennington Biomedical Research Center and CKM is an inventor. There are no other competing interests related to this study to declare.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Höchsmann, C., Martin, C.K. Review of the validity and feasibility of image-assisted methods for dietary assessment. Int J Obes 44, 2358–2371 (2020). https://doi.org/10.1038/s41366-020-00693-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41366-020-00693-2

This article is cited by

Search

Advanced search

Quick links