Letter

Statistically inaccurate and morally unfair judgements via base rate intrusion

Received:
Accepted:
Published online:

Abstract

From a statistical standpoint, judgements about an individual are more accurate if base rates about the individual’s social group are taken into account1,2,3,4. But from a moral standpoint, using these base rates is considered unfair and can even be illegal5,6,7,8,9. Thus, the imperative to be statistically accurate is directly at odds with the imperative to be morally fair. This conflict was resolved by creating tasks in which Bayesian rationality and moral fairness were aligned, thereby allowing social judgements to be both accurate and fair. Despite this alignment, we show that social judgements were inaccurate and unfair. Instead of appropriately setting aside social group differences, participants erroneously relied on them when making judgements about specific individuals. This bias—which we call base rate intrusion—was robust, generalized across various social groups (gender, race, nationality and age), and differed from analogous non-social judgements. Results also demonstrate how social judgements can be corrected to achieve both statistical accuracy and moral fairness. Overall, these data (total N = 5,138) highlight the pernicious effects of social base rates: under conditions that closely approximate those of everyday life10,11,12, these base rates can undermine the rationality and fairness of human judgements.

  • Subscribe to Nature Human Behaviour for full access:

    $99

    Subscribe

Additional access options:

Already a subscriber?  Log in  now or  Register  for online access.

References

  1. 1.

    Eddy, D. M. in Judgment Under Uncertainty: Heuristics and Biases (eds Kahneman, D., Slovic, P. & Tversky, A.) 249–267 (Cambridge Univ. Press, Cambridge, 1982).

  2. 2.

    Kahneman, D. & Tversky, A. On the psychology of prediction. Psychol. Rev. 80, 237–251 (1973).

  3. 3.

    Bar-Hillel, M. The base-rate fallacy in probability judgments. Acta Psychol. 44, 211–233 (1980).

  4. 4.

    Tversky, A. & Kahneman, D. Judgment under uncertainty: heuristics and biases. Science 185, 1124–1131 (1974).

  5. 5.

    Cao, J. & Banaji, M. R. The base rate principle and the fairness principle in social judgment. Proc. Natl Acad. Sci. USA 113, 7475–7580 (2016).

  6. 6.

    Rawls, J. Justice as Fairness: A Restatement (Harvard Univ. Press, Cambridge, 2001).

  7. 7.

    Dworkin, R. Sovereign Virtue: The Theory and Practice of Equality (Harvard Univ. Press, Cambridge, 2000).

  8. 8.

    Koehler, J. in Handbook of Psychology and Law (eds Kagehiro, D. & Laufer, W.) 167–184 (Springer, New York, 1992).

  9. 9.

    Test-Achats v. Council of Ministers (European Court of Justice, 2011).

  10. 10.

    Fiske, S. & Neuberg, S. A continuum of impression formation, from category based to individuating processes: Influences of information and motivation on attention and interpretation. Adv. Exp. Soc. Psychol. 23, 1–74 (1990).

  11. 11.

    Moss-Racusin, C., Dovidio, J., Brescoll, V., Graham, M. & Handelsman, J. Science faculty’s subtle gender biases favor male students. Proc. Natl Acad. Sci. USA 109, 16474–16479 (2012).

  12. 12.

    Cheryan, S., Plaut, C., Davies, P. & Steele, C. Ambient belonging: how stereotypical cues impact gender participation in computer science. J. Pers. Soc. Psychol. 97, 1056–1060 (2009).

  13. 13.

    Locksley, A., Borgida, E., Brekke, N. & Hepburn, C. Sex stereotypes and social judgment. J. Pers. Soc. Psychol. 39, 821–831 (1980).

  14. 14.

    Rasinksi, K. A., Crocker, J. & Hastie, R. Another look at sex stereotypes and social judgments: an analysis of the perceiver’s use of subjective probabilities. J. Pers. Soc. Psychol. 49, 317–326 (1985).

  15. 15.

    Hamilton, D. L. Cognitive Processes in Stereotyping and Intergroup Behavior (Erlbaum, Hillsdale, 1981).

  16. 16.

    Krosnick, J. A., Li, F. & Lehman, D. R. Conversational conventions, order of information acquisition, and the effect of base rates on individuating social information on social judgments. J. Pers. Soc. Psychol. 59, 1140–1152 (1990).

  17. 17.

    Jussim, L. Social Perception and Social Reality: Why Accuracy Dominates Bias and Self-Fulfilling Prophecy (Oxford Univ. Press, Oxford, 2012).

  18. 18.

    Pearl, J. Causality: Models, Reasoning, and Inference (Cambridge Univ. Press, Cambridge, 2000).

  19. 19.

    Hausman, D. M. & Woodard, J. Independence, invariance, and the causal Markov condition. Brit. J. Phil. Sci. 50, 521–583 (1999).

  20. 20.

    Rottman, B. M. & Hastie, R. Reasoning about causal relationships. Inferences on causal networks. Psychol. Bull. 140, 109–139 (2014).

  21. 21.

    Rehder, B. Independence and dependence in human causal reasoning. Cogn. Psychol. 72, 54–107 (2014).

  22. 22.

    Murphy, G. The Big Book of Concepts (MIT, Cambridge, 2002).

  23. 23.

    Medin, D. & Smith, E. Concepts and concept formation. Annu. Rev. Psychol. 35, 113–138 (1984).

  24. 24.

    Banaji, M. & Bhaskar, R. in Memory, Brain, and Belief (eds Schacter, D. & Scarry, E.) 139–175 (Harvard Univ. Press, Cambridge, 1999).

  25. 25.

    Norris, C., Chen, E., Zhu, D., Small, S. & Cacioppo, J. The interaction of social and emotional processes in the brain. J. Cogn. Neurosci. 16, 1818–1829 (2004).

  26. 26.

    Wattenmaker, W. Knowledge structures and linear separability: integrating information in object and social categorization. Cogn. Psychol. 28, 273–328 (1995).

  27. 27.

    Contreras, J., Banaji, M. & Mitchell, J. Dissociable neural correlations of stereotypes and other forms of semantic knowledge. Socl. Cogn. Affect. Neurosci. 7, 764–770 (2012).

  28. 28.

    Mitchell, J., Heatherton, T. & Macrae, C. Distinct neural systems subserve person and object knowledge. Proc. Natl Acad. Sci. USA 99, 15238–15243 (2002).

  29. 29.

    Krynski, T. & Tenenbaum, J. The role of causality in judgment under uncertainty. J. Exp. Psychol. Gen. 126, 430–450 (2007).

  30. 30.

    Kay, A. et al. Inequality, discrimination, and the power of the status quo: direct evidence for a motivation to view what is as what should be. J. Pers. Soc. Psychol. 97, 421–434 (2009).

  31. 31.

    Grice, H. P. in Syntax and Semantics (eds Cole, P. and Morgan, J.) 41–58 (Academic Press, New York, 1975).

  32. 32.

    Villejoubert, G. & Mandel, D. The inverse fallacy: an account of deviations from Bayes’s theorem and the additivity principle. Mem. Cogn. 30, 171–178 (2002).

  33. 33.

    Rottman, B. M. & Hastie, R. Do people reason rationally about causally related events? Markov violations, weak inferences, and failures in explaining away. Cogn. Psychol. 87, 88–134 (2016).

  34. 34.

    Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D. & R Core Team nlme: linear and nonlinear mixed effects models. R package v. 3.1-131 (2017).

  35. 35.

    Bates, D., Maechler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015).

Download references

Acknowledgements

This work was supported by NSF Graduate Research Fellowships to J.C. and M.K.W.; an Inequality and Social Policy fellowship from Harvard University Kennedy School of Government to J.C.; a Hertz Fellowship to M.K.W. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We are grateful to B. Rehder and S. Gershman for helpful comments and to K. Morehouse for research assistance.

Author information

Affiliations

  1. Department of Psychology, Harvard University, Cambridge, MA, 02138, USA

    • Jack Cao
    •  & Mahzarin R. Banaji
  2. Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA

    • Max Kleiman-Weiner

Authors

  1. Search for Jack Cao in:

  2. Search for Max Kleiman-Weiner in:

  3. Search for Mahzarin R. Banaji in:

Contributions

J.C., M.K.W. and M.R.B. designed research. J.C. performed research and analysed data. J.C., M.K.W. and M.R.B. wrote the paper.

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Jack Cao.

Electronic supplementary material

  1. Supplementary Information

    Supplementary Methods, Supplementary Discussion, Supplementary Experiments, Supplementary Figures 1–13, Supplementary Tables 1–7

  2. Life Sciences Reporting Summary

    Life Sciences Reporting Summary