Drivers are blamed more than their automated cars when both make mistakes

Article metrics

Abstract

When an automated car harms someone, who is blamed by those who hear about it? Here we asked human participants to consider hypothetical cases in which a pedestrian was killed by a car operated under shared control of a primary and a secondary driver and to indicate how blame should be allocated. We find that when only one driver makes an error, that driver is blamed more regardless of whether that driver is a machine or a human. However, when both drivers make errors in cases of human–machine shared-control vehicles, the blame attributed to the machine is reduced. This finding portends a public under-reaction to the malfunctioning artificial intelligence components of automated cars and therefore has a direct policy implication: allowing the de facto standards for shared-control vehicles to be established in courts by the jury system could fail to properly regulate the safety of those vehicles; instead, a top-down scheme (through federal laws) may be called for.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Actions or action sequences for the different car types considered.
Fig. 2: Blame ratings for user and industry in six car types.
Fig. 3: Representation of demographic attributes in Study 5.
Fig. 4: Ratings of demographic subgroups in Study 5.

Data availability

Raw data and source data for Figs. 24, Table 1 and Supplementary Fig. 1 are available at https://bit.ly/2kzLymH.

Code availability

Code used to produce figures and tables in this article is available at https://bit.ly/2kzLymH.

References

  1. 1.

    Road traffic injuries. World Health Organization Fact Sheet (WHO, 2017).

  2. 2.

    Geistfeld, M. A. A roadmap for autonomous vehicles: state tort liability, automobile insurance, and federal safety regulation. Calif. L. Rev. 105, 1611 (2017).

  3. 3.

    Tesla. A Tragic Loss https://www.tesla.com/blog/tragic-loss (Tesla, 2016).

  4. 4.

    Automatic Vehicle Control Systems—Investigation of Tesla Accident (National Highway Traffic Safety Administration, 2016).

  5. 5.

    Griswold, A. Uber found not criminally liable in last year’s self-driving car death. Quartz (5 March 2019).

  6. 6.

    Lowy, J. & Krishner, T. Tesla driver killed while using autopilot was watching Harry Potter, witness says. Associated Press News https://apnews.com/ee71bd075fb948308727b4bbff7b3ad8 (30 June 2016).

  7. 7.

    Chong, Z. & Krok, A. Tesla not at fault in fatal crash, driver was not watching a movie. CNET https://www.cnet.com/roadshow/news/tesla-found-not-guilty-of-fatal-crash-says-us-report/ (19 June 2017).

  8. 8.

    Randazzo, R. Who was really at fault in fatal uber crash? here’s the whole story. AZ Central https://www.azcentral.com/story/news/local/tempe/2019/03/17/one-year-after-self-driving-uber-rafaela-vasquez-behind-wheel-crash-death-elaine-herzberg-tempe/1296676002/ (17 March 2019).

  9. 9.

    Munster, G. Here’s when having a self-driving car will be a normal thing. Fortune https://fortune.com/2017/09/13/gm-cruise-self-driving-driverless-autonomous-cars/ (13 September 2017).

  10. 10.

    Kessler, S. A timeline of when self-driving cars will be on the road, according to the people making them. Quartz https://qz.com/943899/a-timeline-of-when-self-driving-cars-will-be-on-the-road-according-to-the-people-making-them/ (29 March 2017).

  11. 11.

    Li, J., Zhao, X., Cho, M.-J., Ju, W. & Malle, B. F. From Trolley to Autonomous Vehicle: Perceptions of Responsibility and Moral Norms in Traffic Accidents with Self-driving Cars SAE Technical Paper 2016-01-0164 https://doi.org/10.4271/2016-01-0164 (SAE, 2016).

  12. 12.

    Chockler, H. & Halpern, J. Y. Responsibility and blame: a structural-model approach. J. Artif. Intell. Res. 22, 93–115 (2004).

  13. 13.

    Gerstenberg, T. & Lagnado, D. A. When contributions make a difference: explaining order effects in responsibility attribution. Psychon. Bull. Rev. 19, 729–736 (2012).

  14. 14.

    Sloman, S. A. & Lagnado, D. Causality in thought. Annu. Rev. Psychol. 66, 223–247 (2015).

  15. 15.

    Zultan, R., Gerstenberg, T. & Lagnado, D. A. Finding fault: causality and counterfactuals in group attributions. Cognition 125, 429–440 (2012).

  16. 16.

    Bonnefon, J.-F., Shariff, A. & Rahwan, I. The social dilemma of autonomous vehicles. Science 352, 1573–1576 (2016).

  17. 17.

    Awad, E. et al. The moral machine experiment. Nature 563, 59 (2018).

  18. 18.

    Malle, B., Scheutz, M., Arnold, T., Voiklis, J. & Cusimano, C. Sacrifice one for the good of many? People apply different. In Proc. 10th ACM/IEEE International Conference on Human-Robot Interaction 117–124 (2015).

  19. 19.

    Shariff, A., Bonnefon, J.-F. & Rahwan, I. Psychological roadblocks to the adoption of self-driving vehicles. Nat. Hum. Behav. 1, 694 (2017).

  20. 20.

    Bornstein, B. H. & Greene, E. Jury decision making: implications for and from psychology. Curr. Dir. Psychol. Sci. 20, 63–67 (2011).

  21. 21.

    Nader, R. Unsafe at Any Speed. The Designed-in Dangers of the American Automobile (Grossman, 1965).

  22. 22.

    Westervelt, E. Did a bail reform algorithm contribute to this San Francisco man’s murder? National Public Radio https://www.npr.org/2017/08/18/543976003/did-a-bail-reform-algorithm-contribute-to-this-san-francisco-man-s-murder (18 August 2017).

  23. 23.

    Bauman, C. W., McGraw, A. P., Bartels, D. M. & Warren, C. Revisiting external validity: concerns about trolley problems and other sacrificial dilemmas in moral psychology. Soc. Personal Psychol. Compass 8, 536–554 (2014).

  24. 24.

    Aronson, E, Wilson, T. D. & Brewer, M. B. in The Handbook of Social Psychology Vol. 1 (eds Gilbert, D. T., Fiske, S. T., & Lindzey, G.) 99–142 (McGraw-Hill, 1998).

  25. 25.

    FeldmanHall, O. et al. Differential neural circuitry and self-interest in real vs hypothetical moral decisions. Soc. Cogn. Affect. Neurosci. 7, 743–751 (2012).

  26. 26.

    Bostyn, D. H., Sevenhant, S. & Roets, A. Of mice, men, and trolleys: hypothetical judgment versus real-life behavior in trolley-style moral dilemmas. Psychol. Sci. 29, 1084–1093 (2018).

  27. 27.

    Dickinson, D. L. & Masclet, D. Using ethical dilemmas to predict antisocial choices with real payoff consequences: an experimental study. J. Econ. Behav. Organ. 166, 195–215 (2018).

  28. 28.

    Plunkett, D. & Greene, J. Overlooked evidence and a misunderstanding of what trolley dilemmas do best: a comment on Bostyn, Sevenhant, & Roets (2018). Psychol. Sci. 30, 1389–1391 (2019).

  29. 29.

    Greene, J. & Haidt, J. How (and where) does moral judgment work? Trends Cogn. Sci. 6, 517–523 (2002).

  30. 30.

    Horberg, E. J., Oveis, C. & Keltner, D. Emotions as moral amplifiers: an appraisal tendency approach to the influences of distinct emotions upon moral judgment. Emot. Rev. 3, 237–244 (2011).

  31. 31.

    Luetge, C. The German ethics code for automated and connected driving. Philos. Technol. 30, 547–558 (2017).

  32. 32.

    Mikhail, J. Elements of Moral Cognition: Rawls’ Linguistic Analogy and the Cognitive Science of Moral and Legal Judgment (Cambridge Univ. Press, 2011).

  33. 33.

    Greene, J. Moral Tribes: Emotion, Reason, and the Gap between Us and Them (Penguin, 2014).

  34. 34.

    Alicke, M. D. Culpable control and the psychology of blame. Psychol. Bull. 126, 556 (2000).

  35. 35.

    Gerstenberg, T., Goodman, N. D., Lagnado, D. A. & Tenenbaum, J. B. How, whether, why: causal judgments as counterfactual contrasts. in Proc. 37th Annual Meeting of the Cognitive Science Society 782–787 (2015).

  36. 36.

    Hitchcock, C. & Knobe, J. Cause and norm. J. Philos. 106, 587–612 (2009).

  37. 37.

    Kominsky, J. F., Phillips, J., Gerstenberg, T., Lagnado, D. & Knobe, J. Causal superseding. Cognition 137, 196–209 (2015).

  38. 38.

    Hart, H. L. A. & Honoré, T. Causation in the Law (Oxford Univ. Press, 1985).

  39. 39.

    Gray, H. M., Gray, K. & Wegner, D. M. Dimensions of mind perception. Science 315, 619–619 (2007).

  40. 40.

    Weisman, K., Dweck, C. S. & Markman, E. M. Rethinking people’s conceptions of mental life. Proc. Natl Acad. Sci. USA 114, 11374–11379 (2017).

  41. 41.

    Gray, K., Young, L. & Waytz, A. Mind perception is the essence of morality. Psychol. Inq. 23, 101–124 (2012).

  42. 42.

    Cushman, F. Crime and punishment: distinguishing the roles of causal and intentional analyses in moral judgment. Cognition 108, 353–380 (2008).

  43. 43.

    Cushman, F. Deconstructing intent to reconstruct morality. Curr. Opin. Psychol. 6, 97–103 (2015).

Download references

Acknowledgements

I.R., E.A., S.L. and S.D. acknowledge support from the Ethics and Governance of Artificial Intelligence Fund. J.-F.B. acknowledges support from the ANR-Labex Institute for Advanced Study in Toulouse, the ANR-3IA Artificial and Natural Intelligence Toulouse Institute and grant no. ANR-17-EURE-0010 from Investissements d’Avenir. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

E.A., S.L., M.K.-W., S.D., J.B.T., A.S., J.-F.B. and I.R. contributed to the conception and design of the research. E.A., S.L., M.K.-W. and S.D. conducted studies. E.A. and J.-F.B. analysed data. S.L., E.A., M.K.-W., J.-F.B. and I.R. wrote the manuscript. All authors reviewed and revised the manuscript.

Correspondence to Joshua B. Tenenbaum or Azim Shariff or Jean-François Bonnefon or Iyad Rahwan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Primary Handling Editor: Mary Elizabeth Sutherland

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Fig. 1 and Supplementary Methods.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Awad, E., Levine, S., Kleiman-Weiner, M. et al. Drivers are blamed more than their automated cars when both make mistakes. Nat Hum Behav (2019) doi:10.1038/s41562-019-0762-8

Download citation