Abstract
Digital letter interventions have proven effective in increasing influenza vaccination rates. In this trial, we sought to further refine these strategies and investigated whether the effectiveness of the strategies could be sustained across consecutive influenza seasons. We enrolled all eligible Danish citizens 65 years of age or older in a nationwide registry-based randomized implementation trial during the 2023–2024 influenza season. Households of participants were randomly assigned in a 2.45:1:1:1:1:1:1 ratio to usual care or six different behaviorally informed electronic letter-based nudges delivered before the influenza vaccination period. The primary endpoint was receipt of influenza vaccination. Statistical analyses accounted for household-level clustering. A total of 881,373 participants (mean age 74.1 ± 6.5 years, 52.1% female) were randomized across 649,487 households. The primary endpoint was met; influenza vaccination rates were higher in the pooled intervention letter group compared to usual care (76.32% versus 76.02%; difference, 0.31 percentage points; 99.29% confidence interval, 0.00–0.61; P = 0.007). Although no individual letter significantly increased influenza vaccination rates, the directionality of effect was consistent across all letters. Effectiveness was particularly pronounced in participants who had not received influenza vaccination during the preceding season (Pinteraction = 0.010). Effectiveness was consistent regardless of whether participants had received a similar electronic letter-based nudge in the preceding season (Pinteraction = 0.26). In summary, electronic letter-based nudges successfully increased influenza vaccination among older adults, and our results suggest that these highly scalable strategies can be implemented effectively and safely across consecutive vaccination seasons.
ClinicalTrials.gov registration: NCT06030726.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Data from the nationwide Danish administrative health registries cannot be made publicly available according to Danish law. Individual-level data can be accessed only through a remote-access secure server environment hosted by the Danish Health Data Authority, and access can be obtained only by researchers employed by the authorized institution that conducted the study. Summarized data can be shared for specific research purposes but cannot be made publicly available. The authors are fully committed to sharing summarized data for, for example, meta-analyses. External researchers who would like to obtain summarized data should contact the corresponding author and include a short written proposal and a data request. Requests will be answered within 1 month. All requests complying with ethical and legal requirements will be granted.
Code availability
Full code cannot be exported from the remote-access secure server environment where data are accessed and all statistical analyses are performed. Select code/command examples are provided in the Supplementary Information document.
References
World Health Organization. Influenza (Seasonal). https://www.who.int/news-room/fact-sheets/detail/influenza-(seasonal) (2023).
Gross, P. A., Hermogenes, A. W., Sacks, H. S., Lau, J. & Levandowski, R. A. The efficacy of influenza vaccine in elderly persons. Ann. Intern. Med. 123, 518–527 (1995).
Rondy, M. et al. Effectiveness of influenza vaccines in preventing severe influenza illness among adults: a systematic review and meta-analysis of test-negative design case–control studies. J. Infect. 75, 381–394 (2017).
McLean, H. Q. Interim estimates of 2022–23 seasonal influenza vaccine effectiveness—Wisconsin, October 2022–February 2023. MMWR Morb. Mortal. Wkly Rep. 72, 201–205 (2023).
Fröbert, O. et al. Influenza vaccination after myocardial infarction: a randomized, double-blind, placebo-controlled, multicenter trial. Circulation 144, 1476–1484 (2021).
Behrouzi, B. et al. Association of influenza vaccination with cardiovascular risk: a meta-analysis. JAMA Netw. Open 5, e228873 (2022).
Modin, D. et al. Influenza vaccine in heart failure. Circulation 139, 575–586 (2019).
Modin, D. et al. Influenza vaccination and cardiovascular events in patients with ischaemic heart disease and heart failure: a meta-analysis. Eur. J. Heart Fail. 25, 1685–1692 (2023).
OECD. Influenza vaccination rates. http://data.oecd.org/healthcare/influenza-vaccination-rates.htm
World Health Organization. Managing seasonal vaccination policies and coverage in the European Region. https://www.who.int/europe/activities/managing-seasonal-vaccination-policies-and-coverage-in-the-european-region
Johansen, N. D., Vaduganathan, M., Bhatt, A. S. & Biering-Sørensen, T. Nudging a nation—the Danish NUDGE trial concept. NEJM Evid. 3, EVIDctw2300024 (2023).
Johansen, N. D. et al. Electronic nudges to increase influenza vaccination uptake in Denmark: a nationwide, pragmatic, registry-based, randomised implementation trial. Lancet 401, 1103–1114 (2023).
Yokum, D., Lauffenburger, J. C., Ghazinouri, R. & Choudhry, N. K. Letters designed with behavioural science increase influenza vaccination in Medicare beneficiaries. Nat. Hum. Behav. 2, 743–749 (2018).
Milkman, K. L. et al. A 680,000-person megastudy of nudges to encourage vaccination in pharmacies. Proc. Natl Acad. Sci. USA 119, e2115126119 (2022).
Szilagyi, P. G. et al. Effect of patient portal reminders sent by a health care system on influenza vaccination rates: a randomized clinical trial. JAMA Intern. Med. 180, 962–970 (2020).
Patel, M. S. et al. A randomized trial of behavioral nudges delivered through text messages to increase influenza vaccination among patients with an upcoming primary care visit. Am. J. Health Promot. 37, 324–332 (2022).
Milkman, K. L. et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proc. Natl Acad. Sci. USA 118, e2101165118 (2021).
Szilagyi, P. G. et al. Text vs patient portal messaging to improve influenza vaccination coverage: a health system–wide randomized clinical trial. JAMA Intern. Med. 184, 519–527 (2024).
European Commission. Influenza. https://health.ec.europa.eu/vaccination/influenza_en
Eurostat. Influenza statistics. https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Influenza_statistics
Statens Serum Institut. Influenza—surveillance. https://www.ssi.dk/sygdomme-beredskab-og-forskning/sygdomsovervaagning/i/influenza-ugens-opgoerelse
Milkman, K. L., Beshears, J., Choi, J. J., Laibson, D. & Madrian, B. C. Using implementation intentions prompts to enhance influenza vaccination rates. Proc. Natl Acad. Sci. USA 108, 10415–10420 (2011).
Brewer, N. T. What works to increase vaccination uptake. Acad. Pediatr. 21, S9–S16 (2021).
Tamer, M. Langt de fleste åbner deres e-Boks hver måned. Så hvorfor får vi nu snart en ny digital postkasse? Politiken https://politiken.dk/forbrugogliv/art8516920/Langt-de-fleste-%C3%A5bner-deres-e-Boks-hver-m%C3%A5ned.-S%C3%A5-hvorfor-f%C3%A5r-vi-nu-snart-en-ny-digital-postkasse (2021).
Johansen, N. D. et al. Rationale and design of NUDGE-FLU-CHRONIC and NUDGE-FLU-2: two nationwide randomized trials of electronic nudges to increase influenza vaccination among patients with chronic diseases and older adults during the 2023/2024 influenza season. Am. Heart J. 272, 23–36 (2024).
Danish Health Authority. Autumn vaccinations against covid-19 and influenza are to help us through the winter. https://www.sst.dk/en/english/news/2022/autumn-vaccinations-against-covid-19-and-influenza-are-to-help-us-through-the-winter (2022).
Dunnett, C. W. New tables for multiple comparisons with a control. Biometrics 20, 482–491 (1964).
Digital Post. https://en.digst.dk/systems/digital-post/
Krause, T. G., Jakobsen, S., Haarh, M. & Mølbak, K. The Danish vaccination register. Euro Surveil. 17, 20155 (2012).
Acknowledgements
The authors would like to thank the Danish Health Data Authority for collaborating on this study.
Author information
Authors and Affiliations
Contributions
N.D.J., M.V., A.S.B., D.M., S.C., B.L.C., K.H.J., C.S.L., L.L., L.W., M.D.-P., L.K., S.D.S., P.S., J.U.S.J., C.J.-M.M., T.G.K. and T.B.-S. made substantial contributions to the conceptualization and design of this study. N.D.J. and T.B.-S. were responsible for data collection. N.D.J., D.M., K.H.J. and T.B.-S. had unrestricted access to and verified the raw data. N.D.J., B.L.C., K.H.J. and T.B.-S. performed the statistical analyses. N.D.J. wrote the first draft of the manuscript. All authors reviewed the manuscript draft and approved the final submitted version.
Corresponding author
Ethics declarations
Competing interests
M.V. has received research grant support or served on advisory boards for American Regent, Amgen, AstraZeneca, Bayer, Baxter Healthcare, Boehringer Ingelheim, Chiesi, Cytokinetics, Lexicon Pharmaceuticals, Novartis, Novo Nordisk, Pharmacosmos, Relypsa, Roche Diagnostics, Sanofi and Tricog Health; has speaker engagements with AstraZeneca, Boehringer Ingelheim, Novartis and Roche Diagnostics; and participates on clinical trial committees for studies sponsored by AstraZeneca, Bayer, Galmed, Occlutech, Novartis and Impulse Dynamics. A.S.B. has received research grant support to his institution from the National Institutes of Health/National Heart, Lung, and Blood Institute, the National Institutes of Health/National Institute on Aging, the American College of Cardiology Foundation and the Centers for Disease Control and Prevention and has received consulting fees from Sanofi Pasteur and Novo Nordisk. B.L.C. has received consulting fees from Amgen, Cardurion, Corvia, Myokardia and Novartis. C.S.L. has received speaker fees and served on advisory boards for GlaxoSmithKline, Merck Sharp & Dohme, Pfizer, Takeda and Valneva. L.K. has received speaker fees from Novo Nordisk, Novartis, AstraZeneca, Boehringer Ingelheim and Bayer. S.D.S. has received research grants from Actelion, Alnylam, Amgen, AstraZeneca, Bellerophon, Bayer, Bristol Myers Squibb, Celladon, Cytokinetics, Eidos, Gilead, GlaxoSmithKline, Ionis, Eli Lilly, Mesoblast, MyoKardia, National Institutes of Health/National Heart, Lung, and Blood Institute, Neurotronik, Novartis, Novo Nordisk, Respicardia, Sanofi, Theracos and US2.AI and has consulted for Abbott, Action, Akros, Alnylam, Amgen, Arena, AstraZeneca, Bayer, Boehringer Ingelheim, Bristol Myers Squibb, Cardior, Cardurion, Corvia, Cytokinetics, Daiichi Sankyo, GlaxoSmithKline, Eli Lilly, Merck, Myokardia, Novartis, Roche, Theracos, Quantum Genomics, Cardurion, Janssen, Cardiac Dimensions, Tenaya, Sanofi, Dinaqor, Tremeau, CellProThera, Moderna, American Regent, Sarepta, Lexicon, Anacardio, Akros and Puretech Health. T.B.-S. has served as chief investigator of the Sanofi-financed NUDGE-FLU trial, the Sanofi-financed DANFLU-1 trial and the Sanofi-financed DANFLU-2 trial; served as steering committee member of the Amgen-financed GALACTIC-HF trial, the Boston Scientific–financed LUX-Dx TRENDS trial and the Boehringer Ingelheim–financed EASi-KIDNEY trial; served on advisory boards for Sanofi, GlaxoSmithKline, Amgen and CSL Squirus; received speaker honoraria from Bayer, GlaxoSmithKline, Novartis, GE Healthcare and Sanofi; consulted for Novo Nordisk, IQVIA and Parexel; and received research grants from GE Healthcare, Sanofi, Boston Scientific, AstraZeneca and Novo Nordisk. All other authors declare no competing interests.
Peer review
Peer review information
Nature Medicine thanks Ivan Fan-Ngai Hung and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Lorenzo Righetto, in collaboration with the Nature Medicine team.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Study timeline.
Overview of the study timeline. The study was conducted during the 2023-2024 influenza season. Letters were delivered to all study arms on September 13, 2023, with an additional reminder letter delivered to the repeated letter arm on September 29, 2023. Participants were followed for the primary endpoint of receipt of influenza vaccination from intervention delivery until January 1, 2024. The vaccination period within the Danish governmental vaccination program began on October 1, 2023. The follow-up period for additional exploratory clinical outcomes ends on May 31, 2024.
Extended Data Fig. 2 Additional subgroup analyses for comparison of pooled intervention letter group vs. usual care.
We tested for effect modification across additional subgroups for the comparison of all intervention arms pooled vs. usual care. Estimates are absolute differences in percentage points with error bars representing 99.29% confidence intervals from binomial regression models with household-level clustered standard errors, identity link, and interaction terms, from which interaction p-values were also derived. Interaction p-values < 0.05 were considered nominally statistically significant. Presence of at least one chronic disease was defined as chronic lung disease, diabetes, chronic cardiovascular disease, cancer, chronic kidney disease, immunodeficiency, neurological/neuromuscular disease, liver disease and/or rheumatic disease. CI, confidence interval.
Extended Data Fig. 3 Standard letter template.
English translation of the standard letter template. Text-based nudges used in several intervention arms were added to the highlighted placeholder. Table 2 shows an overview of the intervention arms including English translation of the specific nudges. All letters were written in Danish language.
Supplementary information
Supplementary Information
NUDGE-FLU-2 study group members, Supplemental Tables 1–4, Stata code for statistical models, study protocol and statistical analysis plan
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Johansen, N.D., Vaduganathan, M., Bhatt, A.S. et al. Electronic nudges for sustained influenza vaccination uptake in older adults: the nationwide randomized NUDGE-FLU-2 trial. Nat Med (2024). https://doi.org/10.1038/s41591-024-03202-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41591-024-03202-4
