Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Statistically inaccurate and morally unfair judgements via base rate intrusion

Abstract

From a statistical standpoint, judgements about an individual are more accurate if base rates about the individual’s social group are taken into account1,2,3,4. But from a moral standpoint, using these base rates is considered unfair and can even be illegal5,6,7,8,9. Thus, the imperative to be statistically accurate is directly at odds with the imperative to be morally fair. This conflict was resolved by creating tasks in which Bayesian rationality and moral fairness were aligned, thereby allowing social judgements to be both accurate and fair. Despite this alignment, we show that social judgements were inaccurate and unfair. Instead of appropriately setting aside social group differences, participants erroneously relied on them when making judgements about specific individuals. This bias—which we call base rate intrusion—was robust, generalized across various social groups (gender, race, nationality and age), and differed from analogous non-social judgements. Results also demonstrate how social judgements can be corrected to achieve both statistical accuracy and moral fairness. Overall, these data (total N = 5,138) highlight the pernicious effects of social base rates: under conditions that closely approximate those of everyday life10,11,12, these base rates can undermine the rationality and fairness of human judgements.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Two Bayesian networks that are identical in structure but differ in whether the base rates are social or non-social.
Fig. 2: The erroneous influence of base rates in experiment 1 (N = 399).
Fig. 3: Social base rates no longer influenced judgments in experiment 4 (N = 411).

References

  1. Eddy, D. M. in Judgment Under Uncertainty: Heuristics and Biases (eds Kahneman, D., Slovic, P. & Tversky, A.) 249–267 (Cambridge Univ. Press, Cambridge, 1982).

  2. Kahneman, D. & Tversky, A. On the psychology of prediction. Psychol. Rev. 80, 237–251 (1973).

    Article  Google Scholar 

  3. Bar-Hillel, M. The base-rate fallacy in probability judgments. Acta Psychol. 44, 211–233 (1980).

    Article  Google Scholar 

  4. Tversky, A. & Kahneman, D. Judgment under uncertainty: heuristics and biases. Science 185, 1124–1131 (1974).

    Article  CAS  PubMed  Google Scholar 

  5. Cao, J. & Banaji, M. R. The base rate principle and the fairness principle in social judgment. Proc. Natl Acad. Sci. USA 113, 7475–7580 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Rawls, J. Justice as Fairness: A Restatement (Harvard Univ. Press, Cambridge, 2001).

    Google Scholar 

  7. Dworkin, R. Sovereign Virtue: The Theory and Practice of Equality (Harvard Univ. Press, Cambridge, 2000).

    Google Scholar 

  8. Koehler, J. in Handbook of Psychology and Law (eds Kagehiro, D. & Laufer, W.) 167–184 (Springer, New York, 1992).

  9. Test-Achats v. Council of Ministers (European Court of Justice, 2011).

  10. Fiske, S. & Neuberg, S. A continuum of impression formation, from category based to individuating processes: Influences of information and motivation on attention and interpretation. Adv. Exp. Soc. Psychol. 23, 1–74 (1990).

    Article  Google Scholar 

  11. Moss-Racusin, C., Dovidio, J., Brescoll, V., Graham, M. & Handelsman, J. Science faculty’s subtle gender biases favor male students. Proc. Natl Acad. Sci. USA 109, 16474–16479 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Cheryan, S., Plaut, C., Davies, P. & Steele, C. Ambient belonging: how stereotypical cues impact gender participation in computer science. J. Pers. Soc. Psychol. 97, 1056–1060 (2009).

    Article  Google Scholar 

  13. Locksley, A., Borgida, E., Brekke, N. & Hepburn, C. Sex stereotypes and social judgment. J. Pers. Soc. Psychol. 39, 821–831 (1980).

    Article  Google Scholar 

  14. Rasinksi, K. A., Crocker, J. & Hastie, R. Another look at sex stereotypes and social judgments: an analysis of the perceiver’s use of subjective probabilities. J. Pers. Soc. Psychol. 49, 317–326 (1985).

    Article  Google Scholar 

  15. Hamilton, D. L. Cognitive Processes in Stereotyping and Intergroup Behavior (Erlbaum, Hillsdale, 1981).

  16. Krosnick, J. A., Li, F. & Lehman, D. R. Conversational conventions, order of information acquisition, and the effect of base rates on individuating social information on social judgments. J. Pers. Soc. Psychol. 59, 1140–1152 (1990).

    Article  Google Scholar 

  17. Jussim, L. Social Perception and Social Reality: Why Accuracy Dominates Bias and Self-Fulfilling Prophecy (Oxford Univ. Press, Oxford, 2012).

    Book  Google Scholar 

  18. Pearl, J. Causality: Models, Reasoning, and Inference (Cambridge Univ. Press, Cambridge, 2000).

    Google Scholar 

  19. Hausman, D. M. & Woodard, J. Independence, invariance, and the causal Markov condition. Brit. J. Phil. Sci. 50, 521–583 (1999).

    Article  Google Scholar 

  20. Rottman, B. M. & Hastie, R. Reasoning about causal relationships. Inferences on causal networks. Psychol. Bull. 140, 109–139 (2014).

    Article  PubMed  Google Scholar 

  21. Rehder, B. Independence and dependence in human causal reasoning. Cogn. Psychol. 72, 54–107 (2014).

    Article  PubMed  Google Scholar 

  22. Murphy, G. The Big Book of Concepts (MIT, Cambridge, 2002).

  23. Medin, D. & Smith, E. Concepts and concept formation. Annu. Rev. Psychol. 35, 113–138 (1984).

    Article  CAS  PubMed  Google Scholar 

  24. Banaji, M. & Bhaskar, R. in Memory, Brain, and Belief (eds Schacter, D. & Scarry, E.) 139–175 (Harvard Univ. Press, Cambridge, 1999).

  25. Norris, C., Chen, E., Zhu, D., Small, S. & Cacioppo, J. The interaction of social and emotional processes in the brain. J. Cogn. Neurosci. 16, 1818–1829 (2004).

    Article  PubMed  Google Scholar 

  26. Wattenmaker, W. Knowledge structures and linear separability: integrating information in object and social categorization. Cogn. Psychol. 28, 273–328 (1995).

    Article  Google Scholar 

  27. Contreras, J., Banaji, M. & Mitchell, J. Dissociable neural correlations of stereotypes and other forms of semantic knowledge. Socl. Cogn. Affect. Neurosci. 7, 764–770 (2012).

    Article  Google Scholar 

  28. Mitchell, J., Heatherton, T. & Macrae, C. Distinct neural systems subserve person and object knowledge. Proc. Natl Acad. Sci. USA 99, 15238–15243 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Krynski, T. & Tenenbaum, J. The role of causality in judgment under uncertainty. J. Exp. Psychol. Gen. 126, 430–450 (2007).

    Article  Google Scholar 

  30. Kay, A. et al. Inequality, discrimination, and the power of the status quo: direct evidence for a motivation to view what is as what should be. J. Pers. Soc. Psychol. 97, 421–434 (2009).

    Article  PubMed  Google Scholar 

  31. Grice, H. P. in Syntax and Semantics (eds Cole, P. and Morgan, J.) 41–58 (Academic Press, New York, 1975).

  32. Villejoubert, G. & Mandel, D. The inverse fallacy: an account of deviations from Bayes’s theorem and the additivity principle. Mem. Cogn. 30, 171–178 (2002).

    Article  Google Scholar 

  33. Rottman, B. M. & Hastie, R. Do people reason rationally about causally related events? Markov violations, weak inferences, and failures in explaining away. Cogn. Psychol. 87, 88–134 (2016).

    Article  PubMed  Google Scholar 

  34. Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D. & R Core Team nlme: linear and nonlinear mixed effects models. R package v. 3.1-131 (2017).

  35. Bates, D., Maechler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015).

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by NSF Graduate Research Fellowships to J.C. and M.K.W.; an Inequality and Social Policy fellowship from Harvard University Kennedy School of Government to J.C.; a Hertz Fellowship to M.K.W. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We are grateful to B. Rehder and S. Gershman for helpful comments and to K. Morehouse for research assistance.

Author information

Authors and Affiliations

Authors

Contributions

J.C., M.K.W. and M.R.B. designed research. J.C. performed research and analysed data. J.C., M.K.W. and M.R.B. wrote the paper.

Corresponding author

Correspondence to Jack Cao.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Supplementary Information

Supplementary Methods, Supplementary Discussion, Supplementary Experiments, Supplementary Figures 1–13, Supplementary Tables 1–7

Life Sciences Reporting Summary

Life Sciences Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, J., Kleiman-Weiner, M. & Banaji, M.R. Statistically inaccurate and morally unfair judgements via base rate intrusion. Nat Hum Behav 1, 738–742 (2017). https://doi.org/10.1038/s41562-017-0218-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-017-0218-y

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing