Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Drivers are blamed more than their automated cars when both make mistakes

Abstract

When an automated car harms someone, who is blamed by those who hear about it? Here we asked human participants to consider hypothetical cases in which a pedestrian was killed by a car operated under shared control of a primary and a secondary driver and to indicate how blame should be allocated. We find that when only one driver makes an error, that driver is blamed more regardless of whether that driver is a machine or a human. However, when both drivers make errors in cases of human–machine shared-control vehicles, the blame attributed to the machine is reduced. This finding portends a public under-reaction to the malfunctioning artificial intelligence components of automated cars and therefore has a direct policy implication: allowing the de facto standards for shared-control vehicles to be established in courts by the jury system could fail to properly regulate the safety of those vehicles; instead, a top-down scheme (through federal laws) may be called for.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Actions or action sequences for the different car types considered.
Fig. 2: Blame ratings for user and industry in six car types.
Fig. 3: Representation of demographic attributes in Study 5.
Fig. 4: Ratings of demographic subgroups in Study 5.

Data availability

Raw data and source data for Figs. 24, Table 1 and Supplementary Fig. 1 are available at https://bit.ly/2kzLymH.

Code availability

Code used to produce figures and tables in this article is available at https://bit.ly/2kzLymH.

References

  1. 1.

    Road traffic injuries. World Health Organization Fact Sheet (WHO, 2017).

  2. 2.

    Geistfeld, M. A. A roadmap for autonomous vehicles: state tort liability, automobile insurance, and federal safety regulation. Calif. L. Rev. 105, 1611 (2017).

    Google Scholar 

  3. 3.

    Tesla. A Tragic Loss https://www.tesla.com/blog/tragic-loss (Tesla, 2016).

  4. 4.

    Automatic Vehicle Control Systems—Investigation of Tesla Accident (National Highway Traffic Safety Administration, 2016).

  5. 5.

    Griswold, A. Uber found not criminally liable in last year’s self-driving car death. Quartz (5 March 2019).

  6. 6.

    Lowy, J. & Krishner, T. Tesla driver killed while using autopilot was watching Harry Potter, witness says. Associated Press News https://apnews.com/ee71bd075fb948308727b4bbff7b3ad8 (30 June 2016).

  7. 7.

    Chong, Z. & Krok, A. Tesla not at fault in fatal crash, driver was not watching a movie. CNET https://www.cnet.com/roadshow/news/tesla-found-not-guilty-of-fatal-crash-says-us-report/ (19 June 2017).

  8. 8.

    Randazzo, R. Who was really at fault in fatal uber crash? here’s the whole story. AZ Central https://www.azcentral.com/story/news/local/tempe/2019/03/17/one-year-after-self-driving-uber-rafaela-vasquez-behind-wheel-crash-death-elaine-herzberg-tempe/1296676002/ (17 March 2019).

  9. 9.

    Munster, G. Here’s when having a self-driving car will be a normal thing. Fortune https://fortune.com/2017/09/13/gm-cruise-self-driving-driverless-autonomous-cars/ (13 September 2017).

  10. 10.

    Kessler, S. A timeline of when self-driving cars will be on the road, according to the people making them. Quartz https://qz.com/943899/a-timeline-of-when-self-driving-cars-will-be-on-the-road-according-to-the-people-making-them/ (29 March 2017).

  11. 11.

    Li, J., Zhao, X., Cho, M.-J., Ju, W. & Malle, B. F. From Trolley to Autonomous Vehicle: Perceptions of Responsibility and Moral Norms in Traffic Accidents with Self-driving Cars SAE Technical Paper 2016-01-0164 https://doi.org/10.4271/2016-01-0164 (SAE, 2016).

  12. 12.

    Chockler, H. & Halpern, J. Y. Responsibility and blame: a structural-model approach. J. Artif. Intell. Res. 22, 93–115 (2004).

    Article  Google Scholar 

  13. 13.

    Gerstenberg, T. & Lagnado, D. A. When contributions make a difference: explaining order effects in responsibility attribution. Psychon. Bull. Rev. 19, 729–736 (2012).

    Article  Google Scholar 

  14. 14.

    Sloman, S. A. & Lagnado, D. Causality in thought. Annu. Rev. Psychol. 66, 223–247 (2015).

    Article  Google Scholar 

  15. 15.

    Zultan, R., Gerstenberg, T. & Lagnado, D. A. Finding fault: causality and counterfactuals in group attributions. Cognition 125, 429–440 (2012).

    Article  Google Scholar 

  16. 16.

    Bonnefon, J.-F., Shariff, A. & Rahwan, I. The social dilemma of autonomous vehicles. Science 352, 1573–1576 (2016).

    CAS  Article  Google Scholar 

  17. 17.

    Awad, E. et al. The moral machine experiment. Nature 563, 59 (2018).

    CAS  Article  Google Scholar 

  18. 18.

    Malle, B., Scheutz, M., Arnold, T., Voiklis, J. & Cusimano, C. Sacrifice one for the good of many? People apply different. In Proc. 10th ACM/IEEE International Conference on Human-Robot Interaction 117–124 (2015).

  19. 19.

    Shariff, A., Bonnefon, J.-F. & Rahwan, I. Psychological roadblocks to the adoption of self-driving vehicles. Nat. Hum. Behav. 1, 694 (2017).

    Article  Google Scholar 

  20. 20.

    Bornstein, B. H. & Greene, E. Jury decision making: implications for and from psychology. Curr. Dir. Psychol. Sci. 20, 63–67 (2011).

    Article  Google Scholar 

  21. 21.

    Nader, R. Unsafe at Any Speed. The Designed-in Dangers of the American Automobile (Grossman, 1965).

  22. 22.

    Westervelt, E. Did a bail reform algorithm contribute to this San Francisco man’s murder? National Public Radio https://www.npr.org/2017/08/18/543976003/did-a-bail-reform-algorithm-contribute-to-this-san-francisco-man-s-murder (18 August 2017).

  23. 23.

    Bauman, C. W., McGraw, A. P., Bartels, D. M. & Warren, C. Revisiting external validity: concerns about trolley problems and other sacrificial dilemmas in moral psychology. Soc. Personal Psychol. Compass 8, 536–554 (2014).

    Article  Google Scholar 

  24. 24.

    Aronson, E, Wilson, T. D. & Brewer, M. B. in The Handbook of Social Psychology Vol. 1 (eds Gilbert, D. T., Fiske, S. T., & Lindzey, G.) 99–142 (McGraw-Hill, 1998).

  25. 25.

    FeldmanHall, O. et al. Differential neural circuitry and self-interest in real vs hypothetical moral decisions. Soc. Cogn. Affect. Neurosci. 7, 743–751 (2012).

    Article  Google Scholar 

  26. 26.

    Bostyn, D. H., Sevenhant, S. & Roets, A. Of mice, men, and trolleys: hypothetical judgment versus real-life behavior in trolley-style moral dilemmas. Psychol. Sci. 29, 1084–1093 (2018).

    Article  Google Scholar 

  27. 27.

    Dickinson, D. L. & Masclet, D. Using ethical dilemmas to predict antisocial choices with real payoff consequences: an experimental study. J. Econ. Behav. Organ. 166, 195–215 (2018).

    Article  Google Scholar 

  28. 28.

    Plunkett, D. & Greene, J. Overlooked evidence and a misunderstanding of what trolley dilemmas do best: a comment on Bostyn, Sevenhant, & Roets (2018). Psychol. Sci. 30, 1389–1391 (2019).

    Article  Google Scholar 

  29. 29.

    Greene, J. & Haidt, J. How (and where) does moral judgment work? Trends Cogn. Sci. 6, 517–523 (2002).

    Article  Google Scholar 

  30. 30.

    Horberg, E. J., Oveis, C. & Keltner, D. Emotions as moral amplifiers: an appraisal tendency approach to the influences of distinct emotions upon moral judgment. Emot. Rev. 3, 237–244 (2011).

    Article  Google Scholar 

  31. 31.

    Luetge, C. The German ethics code for automated and connected driving. Philos. Technol. 30, 547–558 (2017).

    Article  Google Scholar 

  32. 32.

    Mikhail, J. Elements of Moral Cognition: Rawls’ Linguistic Analogy and the Cognitive Science of Moral and Legal Judgment (Cambridge Univ. Press, 2011).

  33. 33.

    Greene, J. Moral Tribes: Emotion, Reason, and the Gap between Us and Them (Penguin, 2014).

  34. 34.

    Alicke, M. D. Culpable control and the psychology of blame. Psychol. Bull. 126, 556 (2000).

    CAS  Article  Google Scholar 

  35. 35.

    Gerstenberg, T., Goodman, N. D., Lagnado, D. A. & Tenenbaum, J. B. How, whether, why: causal judgments as counterfactual contrasts. in Proc. 37th Annual Meeting of the Cognitive Science Society 782–787 (2015).

  36. 36.

    Hitchcock, C. & Knobe, J. Cause and norm. J. Philos. 106, 587–612 (2009).

    Article  Google Scholar 

  37. 37.

    Kominsky, J. F., Phillips, J., Gerstenberg, T., Lagnado, D. & Knobe, J. Causal superseding. Cognition 137, 196–209 (2015).

    Article  Google Scholar 

  38. 38.

    Hart, H. L. A. & Honoré, T. Causation in the Law (Oxford Univ. Press, 1985).

  39. 39.

    Gray, H. M., Gray, K. & Wegner, D. M. Dimensions of mind perception. Science 315, 619–619 (2007).

    CAS  Article  Google Scholar 

  40. 40.

    Weisman, K., Dweck, C. S. & Markman, E. M. Rethinking people’s conceptions of mental life. Proc. Natl Acad. Sci. USA 114, 11374–11379 (2017).

    CAS  Article  Google Scholar 

  41. 41.

    Gray, K., Young, L. & Waytz, A. Mind perception is the essence of morality. Psychol. Inq. 23, 101–124 (2012).

    Article  Google Scholar 

  42. 42.

    Cushman, F. Crime and punishment: distinguishing the roles of causal and intentional analyses in moral judgment. Cognition 108, 353–380 (2008).

    Article  Google Scholar 

  43. 43.

    Cushman, F. Deconstructing intent to reconstruct morality. Curr. Opin. Psychol. 6, 97–103 (2015).

    Article  Google Scholar 

Download references

Acknowledgements

I.R., E.A., S.L. and S.D. acknowledge support from the Ethics and Governance of Artificial Intelligence Fund. J.-F.B. acknowledges support from the ANR-Labex Institute for Advanced Study in Toulouse, the ANR-3IA Artificial and Natural Intelligence Toulouse Institute and grant no. ANR-17-EURE-0010 from Investissements d’Avenir. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Affiliations

Authors

Contributions

E.A., S.L., M.K.-W., S.D., J.B.T., A.S., J.-F.B. and I.R. contributed to the conception and design of the research. E.A., S.L., M.K.-W. and S.D. conducted studies. E.A. and J.-F.B. analysed data. S.L., E.A., M.K.-W., J.-F.B. and I.R. wrote the manuscript. All authors reviewed and revised the manuscript.

Corresponding authors

Correspondence to Joshua B. Tenenbaum or Azim Shariff or Jean-François Bonnefon or Iyad Rahwan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Primary Handling Editor: Mary Elizabeth Sutherland

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Fig. 1 and Supplementary Methods.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Awad, E., Levine, S., Kleiman-Weiner, M. et al. Drivers are blamed more than their automated cars when both make mistakes. Nat Hum Behav 4, 134–143 (2020). https://doi.org/10.1038/s41562-019-0762-8

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing