What we can learn from five naturalistic field experiments that failed to shift commuter behaviour

Abstract

Across five field experiments with employees of a large organization (n = 68,915), we examined whether standard behavioural interventions (‘nudges’) successfully reduced single-occupancy vehicle commutes. In Studies 1 and 2, we sent letters and emails with nudges designed to increase carpooling. These interventions failed to increase carpool sign-up or usage. In Studies 3a and 4, we examined the efficacy of other well-established behavioural interventions: non-cash incentives and personalized travel plans. Again, we found no positive effect of these interventions. Across studies, effect sizes ranged from Cohen’s d = −0.01 to d = 0.05. Equivalence testing, using study-specific smallest effect sizes of interest, revealed that the treatment effects observed in four out of five of our experiments were statistically equivalent to zero (P < 0.04). The failure of these well-powered experiments designed to nudge commuting behaviour highlights both the difficulty of changing commuter behaviour and the importance of publishing null results to build cumulative knowledge about how to encourage sustainable travel.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Forest plot illustrating negligible effects of the five studies by showing the standardized effect size (Cohen’s d) and 95% CI associated with each sample.

Data availability

Coarsened data and modified materials are available at https://osf.io/39rja/. Data and materials have been modified to protect the identity of the partner organization and its employees. Additional materials that do not violate these privacy concerns can be provided on request by the authors.

Code availability

All analyses reported in this study used the statistical software R (v.3.6.1). All R files are available publicly at https://osf.io/39rja/.

References

  1. 1.

    Teter, J., Cazzola, P. & Petropoulos, A. Transport: Tracking Clean Energy Progress https://www.iea.org/tcep/transport (2018).

  2. 2.

    IPCC. Special Report on Global Warming of 1.5 °C (eds Masson-Delmotte, V. et al.) (WMO, 2018).

  3. 3.

    US Census Bureau. Average One-Way Commuting Time by Metropolitan Areas https://www.census.gov/library/visualizations/interactive/travel-time.html (2017).

  4. 4.

    US Census Bureau. American Community Survey 1-Year Estimate https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=bkmk (2017).

  5. 5.

    Hilbrecht, M., Smale, B. & Mock, S. Highway to health? Commute time and well-being among Canadian adults. World Leis. J. 56, 151–163 (2014).

  6. 6.

    Hoehner, C., Barlow, C., Allen, P. & Schootman, M. Commuting distance, cardiorespiratory fitness, and metabolic risk. Am. J. Prev. Med. 42, 571–578 (2012).

  7. 7.

    Martin, A., Panter, J., Suhrcke, M. & Ogilvie, D. Impact of changes in mode of travel to work on changes in body mass index: evidence from the British Household Panel Survey. J. Epidemiol. Community Health 69, 753–761 (2015).

  8. 8.

    Nieuwenhuijsen, M. Influence of urban and transport planning and the city environment on cardiovascular disease. Nat. Rev. Cardiol. 15, 432–438 (2018).

  9. 9.

    Ly, K., Sati, S. & Singer, E. A Behavioural Lens on Transportation Systems: The Psychology of Commuter Behaviour and Transportation Choices Research Report Series: Behavioural Economics in Action (Rotman School of Management, University of Toronto, 2017).

  10. 10.

    Applying Behavioural Insights to Transportation Demand Management (Alta Planning and Design and the Behavioural Insights Team, 2018).

  11. 11.

    Thaler, R. & Sunstein, C. Improving Decisions about Health, Wealth, and Happiness (Yale Univ. Press, 2008).

  12. 12.

    Allcott, H. Social norms and energy conservation. J. Public Econ. 95, 1082–1095 (2011).

  13. 13.

    Milkman, K., Beshears, J., Choi, J., Laibson, D. & Madrian, B. Using implementation intentions prompts to enhance influenza vaccination rates. Proc. Natl Acad. Sci. USA 108, 10415–10420 (2011).

  14. 14.

    Sunstein, C. Nudging: A very short guide. J. Consum. Policy 37, 583–588 (2014).

  15. 15.

    Benartzi, S. et al. Should governments invest more in nudging? Psychol. Sci. 28, 1041–1055 (2017).

  16. 16.

    Organization for Economic Cooperation and Development. Behavioural Insights http://www.oecd.org/gov/regulatory-policy/behavioural-insights.htm (2019).

  17. 17.

    Kenyon, S. & Lyons, G. The value of integrated multimodal traveller information and its potential contribution to modal change. Transp. Res. F 6, 1–21 (2003).

  18. 18.

    Simma, A. & Axhausen, K. Structures of commitment in mode use: a comparison of Switzerland, Germany and Great Britain. Transp. Policy 8, 279–288 (2001).

  19. 19.

    Van Exel, N. & Rietveld, P. Could you also have made this trip by another mode? An investigation of perceived travel possibilities of car and train travellers on the main travel corridors to the city of Amsterdam, The Netherlands. Transp. Res. A 43, 374–385 (2009).

  20. 20.

    Levitt, S. & List, J. Viewpoint: on the generalizability of lab behaviour to the field. Can. J. Econ. 40, 347–370 (2007).

  21. 21.

    Service, O. et al. EAST: Four Simple Ways to Apply Behavioral Insights (The Behavioural Insights Team, 2014).

  22. 22.

    Elgaaied-Gambier, L., Monnot, E. & Reniou, F. Using descriptive norm appeals effectively to promote green behavior. J. Bus. Res. 82, 179–191 (2018).

  23. 23.

    Chorus, C., Molin, E. & Van Wee, B. Use and effects of Advanced Traveller Information Services (ATIS): a review of the literature. Transp. Rev. 26, 127–149 (2006).

  24. 24.

    Bertrand, M., Karlan, D., Mullainathan, S., Shafir, E. & Zinman, J. What’s advertising content worth? Evidence from a consumer credit marketing field experiment. Q. J. Econ. 125, 263–305 (2010).

  25. 25.

    Xia, J., Curtin, K., Li, W. & Zhao, Y. A new model for a carpool matching service. PLoS One 10, e0129257 (2015).

  26. 26.

    One-in-three U.S. drivers cannot pay for an unexpected car repair bill. AAA NewsRoom https://newsroom.aaa.com/2017/04/one-three-u-s-drivers-cannot-pay-unexpected-car-repair-bill (2017).

  27. 27.

    Frederick, S., Novemsky, N., Wang, J., Dhar, R. & Nowlis, S. Opportunity cost neglect. J. Consum. Res. 36, 553–561 (2009).

  28. 28.

    Fujii, S. & Kitamura, R. What does a one-month free bus ticket do to habitual drivers? An experimental analysis of habit and attitude change. Transportation 30, 81–95 (2003).

  29. 29.

    Thøgersen, J. Promoting public transport as a subscription service: effects of a free month travel card. Transp. Policy 16, 335–343 (2009).

  30. 30.

    Kahneman, D. & Tversky, A. Prospect theory: an analysis of decision under risk. Econometrica 47, 263 (1979).

  31. 31.

    Homonoff, T. Can small incentives have large effects? The impact of taxes versus bonuses on disposable bag use. Am. Econ. J. Econ. Policy 10, 177–210 (2018).

  32. 32.

    van Essen, M., Thomas, T., van Berkum, E. & Chorus, C. From user equilibrium to system optimum: a literature review on the role of travel information, bounded rationality and non-selfish behaviour at the network and individual levels. Transp. Rev. 36, 527–548 (2016).

  33. 33.

    Lyons, G. The role of information in decision-making with regard to travel. IEE Intell. Transp. Syst. 153, 199 (2006).

  34. 34.

    Chatterjee, K. A comparative evaluation of large-scale personal travel planning projects in England. Transp. Policy 16, 293–305 (2009).

  35. 35.

    Macmillan, A., Hosking, J., Connor, L., Bullen, J. & Ameratunga, C. S. A Cochrane systematic review of the effectiveness of organisational travel plans: improving the evidence base for transport decisions. Transp. Policy 29, 249–256 (2013).

  36. 36.

    Lakens, D., Scheel, A. & Isager, P. Equivalence testing for psychological research: a tutorial. Adv. Methods Pract. Psychol. Sci. 1, 259–269 (2018).

  37. 37.

    Lakens, D. Equivalence tests. Soc. Psychol. Pers. Sci. 8, 355–362 (2017).

  38. 38.

    Funder, D. & Ozer, D. Evaluating effect size in psychological research: sense and nonsense. Adv. Methods Pract. Psychol. Sci. 2, 156–168 (2019).

  39. 39.

    Ajzen, I. From Intentions to actions: a theory of planned behavior. in Action Control (eds Kuhl, J. and Beckmann, J.) 11–39 (Springer, 1985).

  40. 40.

    Heath, Y. & Gifford, R. Extending the theory of planned behavior: predicting the use of public transportation. J. Appl. Soc. Psychol. 32, 2154–2189 (2002).

  41. 41.

    Cheung, S., Chan, D. & Wong, Z. Reexamining the theory of planned behavior in understanding wastepaper recycling. Environ. Behav. 31, 587–612 (1999).

  42. 42.

    Epley, N. & Schroeder, J. Mistakenly seeking solitude. J. Exp. Psychol. Gen. 143, 1980–1999 (2014).

  43. 43.

    Zhao, Z. & Zhao, J. Car pride and its behavioral implications: an exploration in Shanghai. Transportation https://doi.org/10.1007/s11116-018-9917-0 (2018).

  44. 44.

    Hagman, O. Mobilizing meanings of mobility: car users’ constructions of the goods and bads of car use. Transp. Res. D 8, 1–9 (2003).

  45. 45.

    Steg, L. Car use: lust and must. Instrumental, symbolic and affective motives for car use. Transp. Res. A 39, 147–162 (2005).

  46. 46.

    Oliver, A. Nudging, shoving, and budging: behavioral economic – informed policy. Public Admin. 93, 700–714 (2015).

  47. 47.

    Furman, J. Applying behavioral sciences in the service of four major economic problems. Behav. Sci. Pol. 2, 1–9 (2016).

  48. 48.

    Wilson, R. Estimating the travel and parking demand effects of employer-paid parking. Reg. Sci. Urban Econ. 22, 133–145 (1992).

  49. 49.

    Washbrook, K., Haider, W. & Jaccard, M. Estimating commuter mode choice: a discrete choice analysis of the impact of road pricing and parking charges. Transportation 33, 621–639 (2006).

  50. 50.

    Zahabi, S., Miranda-Moreno, L., Patterson, Z. & Barla, P. Evaluating the effects of land use and strategies for parking and transit supply on mode choice of downtown commuters. J. Transp. Land Use 5, 103–119 (2012).

  51. 51.

    Bartle, C. & Avineri, E. Personalised travel plans in the workplace: a case study. Proc. Inst. Civ. Eng. Munic. Eng. 167, 183–190 (2014).

  52. 52.

    Carroll, G., Choi, J., Laibson, D., Madrian, B. & Metrick, A. Optimal defaults and active decisions. Q. J. Econ. 124, 1639–1674 (2009).

  53. 53.

    Duflo, E., Gale, W., Liebman, J., Orszag, P. & Saez, E. Savings incentives for low- and moderate-income families in the United States: why is the saver’s credit not more effective? J. Eur. Econ. Assoc. 5, 647–661 (2007).

  54. 54.

    B. Long in College Choices: The Economics of Where to Go, When to Go, and How to Pay for It (ed. Hoxby, C. M.) 101–168 (Univ. of Chicago Press, 2004).

  55. 55.

    Bulman, G. & Hoxby, C. The returns to the federal tax credits for higher education. Tax. Policy Econ. 29, 13–88 (2015).

  56. 56.

    Arimura, T., Li, S., Newell, R. & Palmer, K. Cost-effectiveness of electricity energy efficiency programs. Energy J. 33, 63–99 (2012).

  57. 57.

    Brandon, A. et al. Do the effects of social nudges persist? Theory and evidence from 38 natural field experiments. NBER Working Paper No. 23277 https://www.nber.org/papers/w23277 (NBER, 2017).

  58. 58.

    Munafò, M. & Neill, J. Null is beautiful: on the importance of publishing null results. J. Psychopharmacol. 30, 585–585 (2016).

  59. 59.

    Goodchild, L. & Hilten, V. Why it’s time to publish research ‘failures’. Publishing bias favors positive results; now there’s a movement to change that. Elsevier Connect https://www.elsevier.com/connect/scientists-we-want-your-negative-results-too (5 May 2015).

  60. 60.

    Chorus, C. G. A new model of random regret minimization. Eur. J. Transp. Infrastruct. Res. 10, 181–196 (2010).

Download references

Acknowledgements

This project received public funding while one of the researchers was employed at the Behavioral Insights Team, but the authors have not received any funding for the writing and publication of this research. We thank the Behavioral Insights Team for their support in writing this paper. Specifically, we thank M. Sanders and L. Costa. The funders provided input into the design of the study but had no role in data collection and analysis, decision to publish or preparation of this manuscript. We also thank S. O’Flaherty and H. Dystrka for their input, and D. Hagmann, J. Roberts and J. Zlatev. for their comments on earlier drafts of this manuscript.

Author information

Data collection and analysis was performed by A.S.K. A.S.K. drafted the original manuscript. A.V.W. provided critical revisions. Both of the authors approved the final version of the manuscript for submission.

Correspondence to Ariella S. Kristal.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Primary handling editor: Aisha Bradshaw

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Results, Supplementary Notes, Supplementary Discussion, Supplementary Tables 1–4 and Supplementary Fig. 1.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kristal, A.S., Whillans, A.V. What we can learn from five naturalistic field experiments that failed to shift commuter behaviour. Nat Hum Behav (2019). https://doi.org/10.1038/s41562-019-0795-z

Download citation