Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use

Nature Human Behaviour volume 5, pages 1535–1547 (2021)Cite this article

Subjects

Abstract

There is widespread public and academic interest in understanding the uses and effects of digital media. Scholars primarily use self-report measures of the quantity or duration of media use as proxies for more objective measures, but the validity of these self-reports remains unclear. Advancements in data collection techniques have produced a collection of studies indexing both self-reported and log-based measures. To assess the alignment between these measures, we conducted a pre-registered meta-analysis of this research. Based on 106 effect sizes, we found that self-reported media use correlates only moderately with logged measurements, that self-reports were rarely an accurate reflection of logged media use and that measures of problematic media use show an even weaker association with usage logs. These findings raise concerns about the validity of findings relying solely on self-reported measures of media use.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: PRISMA flow diagram for the study inclusion process.
Fig. 2: Forest plot of effect sizes for studies included in the meta-analysis for the association between self-reported and logged measures of digital media use.
Fig. 3: Contour-enhanced funnel plots.
Fig. 4: Forest plot of effect sizes for studies included in the meta-analysis for the association between self-reported and logged problematic media use.
Fig. 5: Forest plot of effect sizes for studies included in the meta-analysis for the ratio of means between self-reported and logged measures of digital media use.

Similar content being viewed by others

Data availability

The raw and processed data are available on the Open Science Framework website (https://osf.io/dhx48/). These data include all extracted effect sizes, study descriptives and descriptive statistics. In cases where raw data were provided by study authors, as with all included studies, we only provide the necessary descriptive statistics and effective sizes used to compute the summary statistics in the meta-analyses, but do not share these original authors’ data. The data have been assigned a unique identifier: https://doi.org/10.17605/OSF.IO/JS6YE.

Code availability

The code (written in the R statistical language) used to analyse the relevant data is provided on the Open Science Framework website (https://osf.io/dhx48/). All materials needed to reproduce the analyses are available at this link.

References

  1. Dickson, K. et al. Screen-Based Activities and Children and Young People’s Mental Health and Psychosocial Wellbeing: A Systematic Map of Reviews (EPPI-Centre, Univ. College London, 2019).

  2. Twenge, J. M. Have smartphones destroyed a generation? The Atlantic. https://www.theatlantic.com/magazine/archive/2017/09/has-the-smartphone-destroyed-a-generation/534198/ (September 2017).

  3. Orben, A. The Sisyphean cycle of technology panics. Perspect Psychol. Sci. 15, 1143–1157 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Orben, A. & Przybylski, A. K. The association between adolescent well-being and digital technology use. Nat. Hum. Behav. 3, 173 (2019).

    Article  PubMed  Google Scholar 

  5. Scharkow, M. The accuracy of self-reported internet use—a validation study using client log data. Commun. Methods Measures 10, 13–27 (2016).

    Article  Google Scholar 

  6. House of Commons Science and Technology Committee. 14th Report - Impact of Social Media and Screen-Use on Young People’s Health https://tinyurl.com/y5omat3s (2019).

  7. Allen, R. L. The reliability and stability of television exposure. Commun. Res. 8, 233–256 (1981).

    Article  Google Scholar 

  8. Bartels, L. M. Messages received: the political impact of media exposure. Am. Political Sci. Rev. 87, 267–285 (1993).

    Article  Google Scholar 

  9. Price, V. The impact of varying reference periods in survey questions about media use. Journal. Q. 70, 615–627 (1993).

    Article  Google Scholar 

  10. Collopy, F. Biases in retrospective self-reports of time use: an empirical study of computer users. Manag. Sci. 42, 758–767 (1996).

    Article  Google Scholar 

  11. Greenberg, B. et al. Comparing survey and diary measures of internet and traditional media use. Commun. Rep. 18, 1–8 (2005).

    Article  Google Scholar 

  12. Coromina, L. & Saris, W. E. Quality of media use measurement. Int. J. Public Opin. Res. 21, 424–450 (2009).

    Article  Google Scholar 

  13. Gerpott, T. J. & Thomas, S. Empirical research on mobile internet usage: a meta-analysis of the literature. Telecommun. Policy 38, 291–310 (2014).

    Article  Google Scholar 

  14. Howard, M. C. & Jayne, B. S. An analysis of more than 1,400 articles, 900 scales, and 17 years of research: the state of scales in cyberpsychology, behavior, and social networking. Cyberpsychol. Behav. Soc. Netw. 18, 181–187 (2015).

    Article  PubMed  Google Scholar 

  15. Griffioen, N., van Rooij, M., Lichtwarck-Aschoff, A. & Granic, I. Toward improved methods in social media research. Technol. Mind Behav. https://doi.org/10.1037/tmb0000005 (2020).

  16. Jungselius, B., & Weilenmann, A. Conceptualizing ‘use’ in social media studies. In Proc. 9th Int. Conf. Social Media Soc. (pp. 325–329) (2018).

  17. Humphreys, L., Karnowski, V. & von Pape, T. Smartphones as metamedia: a framework for identifying the niches structuring smartphone use. Int. J. Commun. 12, 2793–2809 (2018).

    Google Scholar 

  18. Abendroth, A., Parry, D. A., le Roux, D. B., & Gundlach, J. An analysis of problematic media use and technology use addiction scales–what are they actually assessing? in Conference on e-Business, e-Services and e-Society 211–222 (Springer, 2020).

  19. Davidson, B. I., Shaw, H., & Ellis, D. A. Fuzzy constructs in assessment: the overlap between mental health and technology ‘use’. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/6durk (2020).

  20. Flake, J. K. & Fried, E. I. Measurement schmeasurement: questionable measurement practices and how to avoid them. Adv. Methods. Pract. Psychol. 3, 456–465 (2020).

    Google Scholar 

  21. Ellis, D. A., Davidson, B. I., Shaw, H. & Geyer, K. Do smartphone usage scales predict behavior? Int. J. Hum. Comput. Stud. 130, 86–92 (2019).

    Article  Google Scholar 

  22. Ellis, D. A. Are smartphones really that bad? improving the psychological measurement of technology-related behaviors. Comput. Hum. Behav. 97, 60–66 (2019).

    Article  Google Scholar 

  23. Lemmens, J. S., Valkenburg, P. M. & Peter, J. Development and validation of a game addiction scale for adolescents. Media Psychol. 12, 77–95 (2009).

    Article  Google Scholar 

  24. Jobe, J. B. Cognitive psychology and self-reports: models and methods. Qual. Life Res. 12, 219–227 (2003).

    Article  PubMed  Google Scholar 

  25. Sudman, S., Bradburn, N. M., & Schwarz, N. Thinking about Answers: The Application of Cognitive Processes to Survey Methodology (Jossey-Bass, 1996).

  26. Schwarz, N. & Oyserman, D. Asking questions about behavior: cognition, communication, and questionnaire construction. Am. J. Eval. 22, 127–160 (2001).

    Article  Google Scholar 

  27. Tourangeau, R., et al. Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines: Report of the Advanced Research Seminar on Cognitive Aspects of Survey Methodology (National Academic, 1984).

  28. Belson, W. A. The Design and Understanding of Survey Questions (Gower, 1981).

    Google Scholar 

  29. Mieczkowski, H., Lee, A. Y. & Hancock, J. T. Priming effects of social media use scales on well-being outcomes: the influence of intensity and addiction scales on self-reported depression. Soc. Media+ Soc. 6, 2056305120961784 (2020).

    Google Scholar 

  30. Neisser, U. Nested structure in autobiographical memory. in Autobiographical Memory (ed. Rubin D. C.) 71–88 (Cambridge Univ. Press, 1986).

  31. Sewall, C. J., Bear, T. M., Merranko, J. & Rosen, D. How psychosocial well-being and usage amount predict inaccuracies in retrospective estimates of digital technology use. Mob. Media Commun. 8, 379–399 (2020).

    Article  Google Scholar 

  32. Andrews, S., Ellis, D. A., Shaw, H. & Piwek, L. Beyond self-report: tools to compare estimated and real-world smartphone use. PLoS ONE 10, e0139004 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  33. Vorderer, P., Krӧmer, N. & Schneider, F. M. Permanently online–permanently connected: explorations into university students’ use of social media and mobile smart devices. Comput. Hum. Behav. 63, 694–703 (2016).

    Article  Google Scholar 

  34. Piewk, L., Ellis, D. A. & Andrews, S. Can programming frameworks bring smartphones into the mainstream of psychological science? Front. Psychol. 7, 1252 (2016).

    Google Scholar 

  35. Ryding, F. C. & Kuss, D. J. Passive objective measures in the assessment of problematic smartphone use: a systematic review. Addict. Behav. Rep. 11, 100257 (2020).

    PubMed  PubMed Central  Google Scholar 

  36. Boase, J. & Ling, R. Measuring mobile phone use: self-report versus log data. J. Comput.-Mediat. Commun. 18, 508–519 (2013).

    Article  Google Scholar 

  37. Vanden Abeele, M., Beullens, K. & Roe, K. Measuring mobile phone use: gender, age and real usage level in relation to the accuracy and validity of self-reported mobile phone use. Mob. Media Commun. 1, 213–236 (2013).

    Article  Google Scholar 

  38. Kobayashi, T. & Boase, J. No such effect? The implications of measurement error in self-report measures of mobile communication use. Commun. Methods Measures 6, 126–143 (2012).

    Article  Google Scholar 

  39. Deng, T. et al. Measuring smartphone usage and task switching with log tracking and self-reports. Mob. Media Commun. 7, 3–23 (2019).

    Article  Google Scholar 

  40. Jürgens, P., Stark, B. & Magin, M. Two half-truths make a whole? On bias in self-reports and tracking data. Soc. Sci. Comput. Rev. 38, 600–615 (2019).

    Article  Google Scholar 

  41. Ohme, J., Araujo, T., de Vreese, C. H. & Piotrowski, J. T. Mobile data donations: assessing self-report accuracy and sample biases with the iOS Screen Time function. Mob. Media Commun. 9, 293–313 (2021).

    Article  Google Scholar 

  42. Parry, D., et al. Associations between log-based and self-reported digital media use: a systematic review and meta-analysis protocol. Preprint at OSF Preprints https://osf.io/xrg5c (2020).

  43. Araujo, T., Wonneberger, A., Neijens, P. & de Vreese, C. How much time do you spend online? Understanding and improving the accuracy of self-reported measures of Internet use. Commun. Methods Measures 11, 173–190 (2017).

    Article  Google Scholar 

  44. Burke, M., Marlow, C., & Lento, T. Social network activity and social well-being. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 1909–1912 (2010).

  45. Elhai, J. et al. Depression and emotion regulation predict objective smartphone use measured over one week. Personal. Individ. Differences 133, 21–28 (2018).

    Article  Google Scholar 

  46. Ernala, S. K., Burke, M., Leavitt, A., & Ellison, N. B. How well do people report time spent on Facebook? An evaluation of established survey questions with recommendations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems 1–14 (2020).

  47. Felisoni, D. D. & Godoi, A. S. Cell phone usage and academic performance: an experiment. Comput. Educ. 117, 175–187 (2018).

    Article  Google Scholar 

  48. Inyang, I., Benke, G., Morrissey, J., McKenzie, R. & Abramson, M. How well do adolescents recall use of mobile telephones? Results of a validation study. BMC Med. Res. Method. 9, 36 (2009).

    Article  Google Scholar 

  49. Jones-Jang, S. M. et al. Good news! communication findings may be underestimated: comparing effect sizes with self-reported and logged smartphone use data. J. Comput.-Mediat. Commun. 25, 346–363 (2020).

    Article  Google Scholar 

  50. Junco, R. Comparing actual and self-reported measures of Facebook use. Comput. Hum. Behav. 29, 626–631 (2013).

    Article  Google Scholar 

  51. Kahn, A. S., Ratan, R. & Williams, D. Why we distort in self-report: predictors of self-report errors in video game play. J. Comput.-Mediat. Commun. 19, 1010–1023 (2014).

    Article  Google Scholar 

  52. Katapally, T. R. & Chu, L. M. Methodology to derive objective screen-state from smartphones: A SMART platform study. Int. J. Environ. Res. Public Health 16, 2275 (2019).

    Article  PubMed Central  Google Scholar 

  53. Lee, H., Ahn, H., Choi, S. & Choi, W. The SAMS: smartphone addiction management system and verification. J. Med. Syst. 38 (2014).

  54. Lee, H., Ahn, H., Nguyen, T. G., Choi, S. W. & Kim, D. J. Comparing the self-report and measured smartphone usage of college students: a pilot study. Pyschiatry Invest. 14, 198–204 (2017).

    Article  Google Scholar 

  55. Lin, Y. et al. Time distortion associated with smartphone addiction: Identifying smartphone addiction via a mobile application (app). J. Psychiatr. Res. 65, 139–145 (2015).

    Article  PubMed  Google Scholar 

  56. Loid, K., Täht, K. & Rozgonjuk, D. Do pop-up notifications regarding smartphone use decrease screen time, phone checking behavior, and self-reported problematic smartphone use? Evidence from a two-month experimental study. Comput. Hum. Behav. 102, 22–30 (2020).

    Article  Google Scholar 

  57. Mikkelsen, S. et al. Validity of questionnaire self-reports on computer, mouse and keyboard usage during a four-week period. Occup. Environ. Med. 64, 541–547 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  58. Montag, C. et al. Recorded behavior as a valuable resource for diagnostics in mobile phone addiction: evidence from psychoinformatics. Behav. Sci. 5, 434–442 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  59. Newell, C. A. Masters dissertation Smartphone and Social Media Use and Its Health Associations (Univ. Texas at Arlington, 2018).

  60. Noë, B. et al. Identifying indicators of smartphone addiction through user–app interaction. Comput. Hum. Behav. 99, 56–65 (2019).

    Article  Google Scholar 

  61. Oeldorf-Hirsch, A., & Chen, Y. Who Cares about Screen Time? Predicting the use of Mobile Phone Tracking Features. Presented at the 70th Annual Conference of the International Communication Association (ICA), Gold Coast, Australia (virtual conference) (2020)

  62. Pan, Y. C., Lin, H. H., Chiu, Y. C., Lin, S. H. & Lin, Y. H. Temporal stability of smartphone use data: determining fundamental time unit and independent cycle. JMIR mHealth uHealth 7, e13421 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  63. Prasad, S. et al. A study of magnitude and psychological correlates of smartphone use in medical students: a pilot study with a novel telemetric approach. Indian J. Psychol. Med. 40, 468–475 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  64. Rosen, L. D. et al. The role of executive functioning and technological anxiety (FOMO) in college course performance as mediated by technology usage and multitasking habits. Educ. Psychol. 24, 14–25 (2017).

    Google Scholar 

  65. Rozgonjuk, D., Levine, J. C., Hall, B. J. & Elhai, J. D. The association between problematic smartphone use, depression and anxiety symptom severity, and objectively measured smartphone use over one week. Comput. Hum. Behav. 87, 10–17 (2018).

    Article  Google Scholar 

  66. Rozgonjuk, D., Pruunsild, P., Jürimäe, K., Schwarz, R. J. & Aru, J. Instagram use frequency is associated with problematic smartphone use, but not with depression and anxiety symptom severity. Mob. Media Commun. 8, 400–418 (2020).

    Article  Google Scholar 

  67. Sela, Y., Zach, M., Amichay-Hamburger, Y., Mishali, M. & Omer, H. Family environment and problematic internet use among adolescents: the mediating roles of depression and fear of missing out. Comput. Hum. Behav. 106, 106226 (2020).

    Article  Google Scholar 

  68. Shaw, H. et al. Quantifying smartphone ‘use’: choice of measurement impacts relationships between ‘usage’ and health. Technol. Mind Behav. https://doi.org/10.1037/tmb0000022 (2020).

  69. Shin, C., & Dey, A. K. Automatically detecting problematic use of smartphones. Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing 335–344 (2013).

  70. Shin, M. & Lee, K. Measuring smartphone usage time is not sufficient to predict smartphone addiction. J. Theor. Appl. Inform. Technol. 95, 5296–5303 (2017).

    Google Scholar 

  71. Shum, M., Kelsh, M. A., Sheppard, A. R. & Zhao, K. An evaluation of self‐reported mobile phone use compared to billing records among a group of engineers and scientists. Bioelectromagnetics 32, 37–48 (2011).

    Article  PubMed  Google Scholar 

  72. Singh, V. K., & Jain, A. Toward harmonizing self-reported and logged social data for understanding human behavior. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems 2233–2238 (2017).

  73. Timotijevic, L., Barnett, J., Shepherd, R. & Senior, V. Factors influencing self‐report of mobile phone use: the role of response prompt, time reference and mobile phone use in recall. Appl. Cogn. Psychol. 23, 664–683 (2009).

    Article  Google Scholar 

  74. Tokola, K., Kurttio, P., Salminen, T. & Auvinen, A. Reducing overestimation in reported mobile phone use associated with epidemiological studies. Bioelectromagnetics 29, 559–563 (2008).

    Article  PubMed  Google Scholar 

  75. van Berkel, N. et al. Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. Int. J. Hum. Comput. Stud. 125, 118–128 (2019).

    Article  Google Scholar 

  76. Wilmer, H. H., Hampton, W. H., Olino, T. M., Olson, I. R. & Chein, J. M. Wired to be connected? Links between mobile technology engagement, intertemporal preference and frontostriatal white matter connectivity. Soc. Cogn. Affect. Neurosci. 14, 367–379 (2019).

    PubMed  PubMed Central  Google Scholar 

  77. Yuan, N. et al. How much do parents actually use their smartphones? Pilot study comparing self-report to passive sensing. Pediatr. Res. 86, 416–418 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  78. Moyer, A. & Finney, J. W. Rating methodological quality: toward improved assessment and investigation. Account. Res. 12, 299–313 (2005).

    Article  PubMed  Google Scholar 

  79. Crowe, M. & Sheppard, L. A review of critical appraisal tools show they lack rigor: alternative tool structure is proposed. J. Clin. Epidemiol. 64, 79–89 (2011).

    Article  PubMed  Google Scholar 

  80. Protogerou, C. & Hagger, M. S. A case for a study quality appraisal in survey studies in psychology. Front. Psychol. 9, 2788 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  81. Protogerou, C. & Hagger, M. S. A checklist to assess the quality of survey studies in psychology. Methods Psychol. 3, 272–299 (2020).

    Article  Google Scholar 

  82. Rodgers, M. A., & Pustejovsky, J. E. Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes. Psychol. Methods https://doi.org/10.1037/met0000300 (2020).

  83. Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48 (2010).

    Article  Google Scholar 

  84. Geyer, K., Ellis, D. A., Shaw, H., & Davidson, B. I. Open source smartphone apps and tools for measuring, quantifying, and visualizing screen time. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/eqhfa (2020).

  85. Nunnally, J. Psychometric Theory 3rd edn (McGraw-Hill Education, 1994).

  86. Carlson, K. D. & Herdman, A. O. Understanding the impact of convergent validity on research results. Organ. Res. Methods 15, 17–32 (2012).

    Article  Google Scholar 

  87. Social Media Addiction Reduction Technology Act, S. 2314, 116th Congress. https://www.congress.gov/bill/116th-congress/senate-bill/2314/all-info (2019).

  88. Salganik, M. J. Bit by Bit: Social Research in the Digital Age (Princeton Univ. Press, 2019).

  89. Sen, I., Flöck, F., Weller, K., Weiß, B., & Wagner, C. From the total survey error framework to an error framework for digital traces of humans: translation tutorial. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency https://doi.org/10.1145/3351095.3375669 (2020).

  90. Dienlin, T. et al. An agenda for open science in communication. J. Commun. 71, 1–26 (2020).

    Google Scholar 

  91. Bishara, A. J. & Hittner, J. B. Reducing bias and error in the correlation coefficient due to nonnormality. Educ. Psycholog. Meas. 75, 785–804 (2015).

    Article  Google Scholar 

  92. Lazer, D. M. et al. Computational social science: obstacles and opportunities. Science 369, 1060–1062 (2020).

    Article  CAS  PubMed  Google Scholar 

  93. Sewall, C. J., & Parry, D. A., What is the role of depression in the discrepancy between estimated and actual iPhone use? An exploratory analysis using cubic response surface analysis. Preprint at OSF Preprints https://doi.org/10.31219/osf.io/e9pfa (2021).

  94. Van Smeden, M., Lash, T. L. & Groenwold, R. H. Reflection on modern methods: five myths about measurement error in epidemiological research. Int. J. Epidemiol. 49, 3444–347 (2020).

    Google Scholar 

  95. Kaye, L. K., Orben, A., Ellis, D. A., Hunger, S. C. & Houghton, S. The conceptual and methodological mayhem of ‘screen time’. Int. J. Environ. Res. Public Health 17, 3661 (2020).

    Article  Google Scholar 

  96. Meier, A. & Reinecke, L. Computer-mediated communication, social media, and mental health: a conceptual and empirical meta-review. Commun. Res. https://doi.org/10.1177/0093650220958224 (2020).

  97. Büchi, M. A proto-theory of digital well-being. Preprint at OSF Preprints https://doi.org/10.31219/osf.io/k3e2j (2020)

  98. Harris, B., Regan, T., Schueler, J. & Fields, S. A. Problematic mobile phone and smartphone use scales: a systematic review. Front. Psychol. 11, 672 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  99. Laconi, S., Rodgers, R. F. & Chabrol, H. The measurement of internet addiction: a critical review of existing scales and their psychometric properties. Comput. Hum. Behav. 41, 190–202 (2014).

    Article  Google Scholar 

  100. Webster, J. & Watson, R. Analyzing the past to prepare for the future: writing a literature review. Manag. Inform. Syst. Q. 26, xiii–xxiii (2002).

    Google Scholar 

  101. Hedges, L. V., Tipton, E. & Johnson, M. C. Robust variance estimation in meta-regression with dependent effect size estimates. Res. Synth. Methods 1, 39–65 (2010).

    Article  PubMed  Google Scholar 

  102. Fisher, Z., & Tipton, E. Robumeta: an R-package for robust variance estimation in meta-analysis. Preprint at arXiv https://arxiv.org/abs/1503.02220 (2015).

  103. Fu, R. et al. Conducting quantitative synthesis when comparing medical interventions: AHRQ and the effective health care program. J. Clin. Epidemiol. 64, 1187–1197 (2011).

    Article  PubMed  Google Scholar 

  104. Gilpin, A. R. Table for conversion of Kendall’s tau to Spearman’s rho within the context of measures of magnitude of effect for meta-analysis. Educ. Psycholog. Meas. 53, 87–92 (1993).

    Article  Google Scholar 

  105. Rupinski, M. T. & Dunlap, W. P. Approximating Pearson product–moment correlations from Kendall’s tau and Spearman’s rho. Educ. Psychol. Meas. 56, 419–429 (1996).

    Article  Google Scholar 

  106. Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. Introduction to Meta-Analysis (Wiley, 2011).

  107. Cohen, J. Statistical Power Analysis for the Behavioral Sciences (Academic, 1988).

  108. Friedrich, J. O., Adhikari, N. K. & Beyene, J. The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: a simulation study. BMC Med. Res. Method. 8, 32 (2008).

    Article  Google Scholar 

  109. Lajeunesse, M. J. On the meta-analysis of response ratios for studies with correlated and multi-group designs. Ecology 92, 2049–2055 (2011).

    Article  Google Scholar 

  110. Egger, M., Smith, G. D., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 315, 629–634 (1997).

    Article  CAS  Google Scholar 

  111. Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R. & Rushton, L. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J. Clin. Epidemiol. 61, 991–996 (2008).

    Article  PubMed  Google Scholar 

  112. Tanner-Smith, E. E., Tipton, E. & Polanin, J. R. Handling complex meta-analytic data structures using robust variance estimates: a tutorial in R. J. Dev. Life-Course Criminol. 2, 85–112 (2016).

    Article  Google Scholar 

  113. Pustejovsky, J. E. clubSandwich: cluster-robust (sandwich) variance estimators with small-sample corrections (R package version 0.4.2) (2017).

Download references

Acknowledgements

The authors thank the researchers who made their data available for inclusion in this review and especially those who completed extra analyses to provide relevant data and statistics. Additionally, the authors thank those who shared our public calls for relevant data and the valuable input provided by the reviewers of this paper. The authors received no specific funding for this work.

Author information

Authors and Affiliations

  1. Department of Information Science, Stellenbosch University, Stellenbosch, South Africa

    Douglas A. Parry

  2. School of Management, University of Bath, Bath, UK

    Brittany I. Davidson

  3. Faculty of Engineering, University of Bristol, Bristol, UK

    Brittany I. Davidson

  4. School of Social Work, University of Pittsburgh, Pittsburgh, PA, USA

    Craig J. R. Sewall

  5. College of Media, University of Illinois Urbana-Champaign, Urbana, IL, USA

    Jacob T. Fisher

  6. Department of Communication, Stanford University, Stanford, CA, USA

    Hannah Mieczkowski

  7. NORMENT, Center for Psychosis Research, Oslo University Hospital and University of Oslo, Oslo, Norway

    Daniel S. Quintana

  8. KG Jebsen Centre for Neurodevelopmental Disorders, University of Oslo, Oslo, Norway

    Daniel S. Quintana

  9. Department of Psychology, University of Oslo, Oslo, Norway

    Daniel S. Quintana

Authors
  1. Douglas A. Parry
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brittany I. Davidson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Craig J. R. Sewall
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jacob T. Fisher
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hannah Mieczkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Daniel S. Quintana
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

D.A.P. and B.I.D. conceived the study. D.A.P., B.I.D., C.J.R.S., J.T.F. and H.M. collected the data. D.A.P. analysed the data with input from D.S.Q. D.A.P., B.I.D., C.J.R.S., J.T.F. and H.M. wrote the first draft of the paper. All authors discussed the results and contributed to revision of the final manuscript.

Corresponding author

Correspondence to Douglas A. Parry.

Ethics declarations

Competing interests

The author(s) declare that there are no conflicts of interest with respect to the authorship or the publication of this article.

Additional information

Peer review information Nature Human Behaviour thanks Anna Chaimani and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Digital media usage post hoc moderator and subgroup analyses.

Note: k: number of separate effect sizes included for the moderator level; r = Pearson correlation coefficient; F values correspond to the Approximate Hotelling-Zhang with small sample correction omnibus tests for moderators with more than two levels; 95% CI corresponds to the r values for individual moderator levels; p corresponds to the F value for moderators or the subgroup analysis for individual moderator levels. *This analysis did not include the adolescent population group as only two effect sizes were available. †This analysis did not include the other category as only a single effect size was available.

Extended Data Fig. 2 Reporting accuracy post hoc moderator and subgroup analyses.

Note: k: number of separate effect sizes included for the moderator level; R = response ratio; Exp(𝛽) = exponential transformation of metaregression coefficient from a model in which a categorical moderator with two levels was entered as a predictor. F values correspond to the Approximate Hotelling-Zhang with small sample correction omnibus tests for moderators with more than two levels; 95% CI corresponds to the r values for individual moderator levels; p corresponds to the F value for moderators or the subgroup analysis for individual moderator levels. *This analysis did not include the adolescent population category, the general population category and the unknown population category as only two, one, and three effect sizes were available, respectively.

Extended Data Fig. 3 Descriptive statistics for additional post hoc moderator analyses.

Note: k: number of included effect sizes. *: One study used both a built-in tool and a third-party tool.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parry, D.A., Davidson, B.I., Sewall, C.J.R. et al. A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nat Hum Behav 5, 1535–1547 (2021). https://doi.org/10.1038/s41562-021-01117-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-021-01117-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing