Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The relationship between political ideology and judgements of bias in distributional outcomes

Nature Human Behaviour volume 8, pages 228–242 (2024)Cite this article

Subjects

Abstract

We examine judgements of bias in distributional outcomes. Such judgements are often based on imbalance in distributional outcomes, namely, the under- or over-representation of a target group relative to some baseline. Using data from 26 studies (N = 14,925), we test how these judgements of bias vary with the target group’s characteristics (traditionally dominant or non-dominant) and the observer’s political ideology (liberal or conservative). We find that conservatives set a higher threshold for recognizing bias against traditionally non-dominant targets (women, Black people, immigrants), as compared with liberals. Conversely, liberals set a higher threshold for recognizing bias against traditionally dominant targets (men, white people, native-born citizens), as compared with conservatives. However, these relationships between political ideology and judgements of bias diminish when the targets are unknown or ideologically irrelevant. These findings emphasize the context-dependency of bias judgements and underscore the importance of stimulus sampling and appropriate selection of controls.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Dynamic updating or the ‘choice titration’ method of measuring bias threshold.
Fig. 2: Correlation between political ideology and bias threshold for hiring decisions by target of bias (Study 1).
Fig. 3: Floodlight analyses probing the interaction between political ideology and bias target (Study 1).
Fig. 4: Floodlight analyses probing the interaction between political ideology and bias target (Study 2).
Fig. 5: A forest plot of correlations between political ideology and bias threshold by target group.
Fig. 6: Floodlight analyses for dominant, non-dominant and unknown targets.

Similar content being viewed by others

Data availability

The datasets analysed in this research can be accessed on the project’s Open Science Framework page84 at https://osf.io/sa35x/.

Code availability

The code for reproducing the analyses in this research can be accessed on the project’s Open Science Framework page84 at https://osf.io/sa35x/.

References

  1. Douthat, R. Google’s war over the sexes. The New York Times (9 August 2017).

  2. Belkin, D. Harvard accused of bias against Asian-Americans. The Wall Street Journal (15 May 2015).

  3. Belson, K. & Vrentas, J. Brian Flores sues N.F.L., claiming bias in coaching search. The New York Times (1 February 2022).

  4. Walster, E. Assignment of responsibility for an accident. J. Pers. Soc. Psychol. 3, 73–79 (1966).

    Article  CAS  PubMed  Google Scholar 

  5. Baron, J. & Hershey, J. C. Outcome bias in decision evaluation. J. Pers. Soc. Psychol. 54, 569–579 (1988).

    Article  CAS  PubMed  Google Scholar 

  6. Lipshitz, R. ‘Either a medal or a corporal’: the effects of success and failure on the evaluation of decision making and decision makers. Organ. Behav. Hum. Decis. Process. 44, 380–395 (1989).

    Article  Google Scholar 

  7. Ditto, P. H. & Lopez, D. F. Motivated skepticism: use of differential decision criteria for preferred and nonpreferred conclusions. J. Pers. Soc. Psychol. 63, 568–584 (1992).

    Article  Google Scholar 

  8. Lord, C. G., Ross, L. & Lepper, M. R. Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence. J. Pers. Soc. Psychol. 37, 2098–2109 (1979).

    Article  Google Scholar 

  9. Vallone, R. P., Ross, L. & Lepper, M. R. The hostile media phenomenon: biased perception and perceptions of media bias in coverage of the Beirut massacre. J. Pers. Soc. Psychol. 49, 577–585 (1985).

    Article  CAS  PubMed  Google Scholar 

  10. Kressel, N. J. Biased judgments of media bias: a case study of the Arab–Israeli dispute. Polit. Psychol. 8, 211–227 (1987).

    Article  Google Scholar 

  11. Hastorf, A. H. & Cantril, H. They saw a game; a case study. J. Abnorm. Soc. Psychol. 49, 129–134 (1954).

    Article  CAS  Google Scholar 

  12. Jost, J. T., Federico, C. M. & Napier, J. L. Political ideology: its structure, functions, and elective affinities. Annu. Rev. Psychol. 60, 307–337 (2009).

    Article  PubMed  Google Scholar 

  13. Milburn, M. A., Conrad, S. D., Sala, F. & Carberry, S. Childhood punishment, denial, and political attitudes. Polit. Psychol. 16, 447–478 (1995).

    Article  Google Scholar 

  14. De Neve, J. E. Personality, childhood experience, and political ideology. Polit. Psychol. 36, 55–73 (2015).

    Article  PubMed  Google Scholar 

  15. Haidt, J. & Graham, J. When morality opposes justice: conservatives have moral intuitions that liberals may not recognize. Soc. Justice Res. 20, 98–116 (2007).

    Article  Google Scholar 

  16. Conway, L. G., Chan, L. & Woodard, S. R. Socio-ecological influences on political ideology. Curr. Opin. Psychol. 32, 76–80 (2020).

    Article  PubMed  Google Scholar 

  17. Niemi, R. G. & Jennings, M. K. Issues and inheritance in the formation of party identification. Am. J. Polit. Sci. 35, 970–988 (1991).

    Article  Google Scholar 

  18. Baron, J. & Jost, J. T. False equivalence: are liberals and conservatives in the United States equally biased? Perspect. Psychol. Sci. 14, 292–303 (2019).

    Article  PubMed  Google Scholar 

  19. Ditto, P. H. et al. At least bias is bipartisan: a meta-analytic comparison of partisan bias in liberals and conservatives. Perspect. Psychol. Sci. 14, 273–291 (2019).

    Article  PubMed  Google Scholar 

  20. Kerlinger, F. N. A social attitude scale: evidence on reliability and validity. Psychol. Rep. 26, 379–383 (1970).

    Article  Google Scholar 

  21. Duckitt, J., Wagner, C., du Plessis, I. & Birum, I. The psychological bases of ideology and prejudice: testing a dual process model. J. Pers. Soc. Psychol. 83, 75–93 (2002).

    Article  PubMed  Google Scholar 

  22. Feldman, S. & Johnston, C. Understanding the determinants of political ideology: implications of structural complexity. Polit. Psychol. 35, 337–358 (2014).

    Article  Google Scholar 

  23. McLellan, D. Ideology (Concepts in Social Thought) (Univ. of Minnesota Press, 1995).

  24. Sowell, T. A Conflict of Visions: Ideological Origins of Political Struggles (Basic Books, 2002).

  25. Haidt, J. The Righteous Mind: Why Good People are Divided by Politics and Religion (Vintage, 2012).

  26. Graham, J., Haidt, J. & Nosek, B. A. Liberals and Conservatives rely on different sets of moral foundations. J. Pers. Soc. Psychol. 96, 1029–1046 (2009).

    Article  PubMed  Google Scholar 

  27. Ridgeway, C. L. & Correll, S. J. Unpacking the gender system a theoretical perspective on gender beliefs and social relations. Gend. Soc. 18, 510–531 (2004).

    Article  Google Scholar 

  28. Axt, J. R., Ebersole, C. R. & Nosek, B. A. The rules of implicit evaluation by race, religion, and age. Psychol. Sci. 25, 1804–1815 (2014).

    Article  PubMed  Google Scholar 

  29. Zou, L. X. & Cheryan, S. Two axes subordination: a new model racial position. J. Pers. Soc. Psychol. 112, 696–717 (2017).

    Article  PubMed  Google Scholar 

  30. Kim, C. J. The racial triangulation of Asian Americans. Polit. Soc. 27, 105–138 (1999).

    Article  Google Scholar 

  31. Quillian, L. Prejudice as a response to perceived group threat: population composition and anti-immigrant and racial prejudice in Europe. Am. Sociol. Rev. 60, 586–611 (1995).

    Article  Google Scholar 

  32. Ruel, E. & Hauser, R. M. Explaining the gender wealth gap. Demography 50, 1155–1176 (2013).

    Article  PubMed  Google Scholar 

  33. Conley, D. Parental resources, inheritance, and investment dynamics. Sociol. Inq. 71, 39–66 (2001).

    Article  Google Scholar 

  34. Berry, J. W. A psychology of immigration. J. Soc. 57, 615–631 (2001).

    Google Scholar 

  35. Knuth, D. E. The Art of Computer Programming, Volume 3: Sorting and Searching (Addison-Wesley Professional, 1998).

  36. Spiller, S. A., Fitzsimons, G. J., Lynch, J. G. & McClelland, G. H. Spotlights, floodlights, and the magic number zero: simple effects tests in moderated regression. J. Mark. Res. 50, 277–288 (2013).

    Article  Google Scholar 

  37. Johnson, P. O. & Neyman, J. Tests of certain linear hypotheses and their application to some educational problems. Stat. Res. Mem. 1, 57–93 (1936).

  38. Aiken, L. S., West, S. G. & Reno, R. R. Multiple Regression: Testing and Interpreting Interactions (Sage, 1991).

  39. Kim, J. Floodlight Analysis Tool (‘FAT’) v0.5: A Tutorial (Open Science Framework, 2022); http://floodlightanalysis.com/floodlight.pdf

  40. Kim, J. kim: A toolkit for behavioral scientists. R package version 0.5.422 https://jinkim.science/docs/kim.pdf (2023).

  41. Hedges, L. V. & Olkin, I. Statistical Methods for Meta-Analysis (Academic Press, 1985).

  42. Field, A. P. Meta-analysis of correlation coefficients: a Monte Carlo comparison of fixed- and random-effects methods. Psychol. Methods 6, 161–180 (2001).

    Article  PubMed  Google Scholar 

  43. Hayes, A. F. & Montoya, A. K. A tutorial on testing, visualizing, and probing an interaction involving a multicategorical variable in linear regression analysis. Commun. Methods Meas. 11, 1–30 (2017).

    Article  Google Scholar 

  44. Koehler, J. J. The influence of prior beliefs on scientific judgments of evidence quality. Organ. Behav. Hum. Decis. Process. 56, 28–55 (1993).

    Article  Google Scholar 

  45. Hahn, U. & Harris, A. J. L. in Psychology of Learning and Motivation Vol. 61 (ed. Ross, B. H.) 41–102 (Elsevier, 2014).

  46. Graham, J. et al. in Advances in Experimental Social Psychology Vol. 47 (eds Devine, P. & Plant, A.) 55–130 (Elsevier, 2013).

  47. Kivikangas, J. M., Fernández-Castilla, B., Järvelä, S., Ravaja, N. & Lönnqvist, J.-E. Moral foundations and political orientation: systematic review and meta-analysis. Psychol. Bull. 147, 55–94 (2021).

    Article  PubMed  Google Scholar 

  48. Voelkel, J. G. & Brandt, M. J. The effect of ideological identification on the endorsement of moral values depends on the target group. Pers. Soc. Psychol. Bull. 45, 851–863 (2019).

    Article  PubMed  Google Scholar 

  49. Kugler, M., Jost, J. T. & Noorbaloochi, S. Another look at moral foundations theory: do authoritarianism and social dominance orientation explain liberal–conservative differences in ‘moral’ intuitions? Soc. Justice Res. 27, 413–431 (2014).

    Article  Google Scholar 

  50. Federico, C. M., Weber, C. R., Ergun, D. & Hunt, C. Mapping the connections between politics and morality: the multiple sociopolitical orientations involved in moral intuition. Polit. Psychol. 34, 589–610 (2013).

    Article  Google Scholar 

  51. Frimer, J. A. Do liberals and conservatives use different moral languages? Two replications and six extensions of Graham, Haidt, and Nosek’s (2009) moral text analysis. J. Res. Pers. 84, 103906 (2020).

    Article  Google Scholar 

  52. Frimer, J. A., Biesanz, J. C., Walker, L. J. & MacKinlay, C. W. Liberals and conservatives rely on common moral foundations when making moral judgments about influential people. J. Pers. Soc. Psychol. 104, 1040–1059 (2013).

    Article  PubMed  Google Scholar 

  53. Waldfogel, H. B., Sheehy-Skeffington, J., Hauser, O. P., Ho, A. K. & Kteily, N. S. Ideology selectively shapes attention to inequality. Proc. Natl Acad. Sci. USA 118, e2023985118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Chambers, J. R., Schlenker, B. R. & Collisson, B. Ideology and prejudice: the role of value conflicts. Psychol. Sci. 24, 140–149 (2013).

    Article  PubMed  Google Scholar 

  55. Brandt, M. J., Reyna, C., Chambers, J. R., Crawford, J. T. & Wetherell, G. The ideological-conflict hypothesis: intolerance among both liberals and conservatives. Curr. Dir. Psychol. Sci. 23, 27–34 (2014).

    Article  Google Scholar 

  56. Sibley, C. G. & Duckitt, J. Personality and prejudice: a meta-analysis and theoretical review. Pers. Soc. Psychol. Rev. 12, 248–279 (2008).

    Article  PubMed  Google Scholar 

  57. Brandt, M. J. & Crawford, J. T. in Advances in Experimental Social Psychology Vol. 61 (ed. Gawronski, B.) 1–66 (Elsevier, 2020).

  58. Crawford, J. T. & Brandt, M. J. Ideological (a)symmetries in prejudice and intergroup bias. Curr. Opin. Behav. Sci. 34, 40–45 (2020).

    Article  Google Scholar 

  59. Crawford, J. T., Jussim, L., Cain, T. R. & Cohen, F. Right-wing authoritarianism and social dominance orientation differentially predict biased evaluations of media reports: dual-process model and media reports. J. Appl. Soc. Psychol. 43, 163–174 (2013).

    Article  Google Scholar 

  60. Tetlock, P. E. Cognitive style and political ideology. J. Pers. Soc. Psychol. 45, 118–126 (1983).

    Article  Google Scholar 

  61. Eidelman, S., Crandall, C. S., Goodman, J. A. & Blanchar, J. C. Low-effort thought promotes political conservatism. Pers. Soc. Psychol. Bull. 38, 808–820 (2012).

    Article  PubMed  Google Scholar 

  62. Wojcik, S. P., Hovasapian, A., Graham, J., Motyl, M. & Ditto, P. H. Conservatives report, but liberals display, greater happiness. Science 347, 1243–1246 (2015).

    Article  ADS  CAS  PubMed  Google Scholar 

  63. Uhlmann, E. L., Pizarro, D. A., Tannenbaum, D. & Ditto, P. H. The motivated use of moral principles. Judgm. Decis. Mak. 4, 479–491 (2009).

    Article  Google Scholar 

  64. Ditto, P. H. et al. Partisan bias and its discontents. Perspect. Psychol. Sci. 14, 304–316 (2019).

    Article  PubMed  Google Scholar 

  65. Nisbet, E. C., Cooper, K. E. & Garrett, R. K. The partisan brain: how dissonant science messages lead conservatives and liberals to (dis)trust science. Ann. Am. Acad. Pol. Soc. Sci. 658, 36–66 (2015).

    Article  Google Scholar 

  66. Strickland, B. & Suben, A. Experimenter philosophy: the problem of experimenter bias in experimental philosophy. Rev. Philos. Psychol. 3, 457–467 (2012).

    Article  Google Scholar 

  67. Wells, G. L. & Windschitl, P. D. Stimulus sampling and social psychological experimentation. Pers. Soc. Psychol. Bull. 25, 1115–1125 (1999).

    Article  Google Scholar 

  68. Monin, B. & Oppenheimer, D. M. The limits of direct replications and the virtues of stimulus sampling. Soc. Psychol. 45, 299–300 (2014).

  69. Hatemi, P. K., Crabtree, C. & Smith, K. B. Ideology justifies morality: political beliefs predict moral foundations. Am. J. Polit. Sci. 63, 788–806 (2019).

    Article  Google Scholar 

  70. R Core Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2022).

  71. Wickham, H. ggplot2: Elegant Graphics for Data Analysis (Springer, 2016).

  72. Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. https://doi.org/10.18637/jss.v036.i03 (2010).

  73. Auguie, B. gridExtra: Miscellaneous functions for ‘grid’ graphics. R package version 2.3 https://cran.r-project.org/package=gridExtra (2017).

  74. Dowle, M. & Srinivasan, A. data.table: Extension of ‘data.frame’. R package version 1.14.6 https://r-datatable.com (2022).

  75. Horowitz, J., Parker, K. & Stepler, R. Wide Partisan Gaps in U.S. over How Far the Country Has Come on Gender Equality (Pew Research Center, 2017).

  76. Horowitz, J. M., Brown, A. & Cox, K. Race in America 2019 (Pew Research Center, 2019).

  77. Dalbert, C. The world is more just for me than generally: about the personal belief in a just world scale’s validity. Soc. Justice Res. 12, 79–98 (1999).

    Article  Google Scholar 

  78. Krijnen, J. M. T., Ülkümen, G., Bogard, J. E. & Fox, C. R. Lay theories of financial well-being predict political and policy message preferences. J. Pers. Soc. Psychol. 122, 310–336 (2022).

  79. Ho, A. K. et al. The nature of social dominance orientation: theorizing and measuring preferences for intergroup inequality using the new SDO7 scale. J. Pers. Soc. Psychol. 109, 1003–1028 (2015).

    Article  PubMed  Google Scholar 

  80. Pratto, F., Sidanius, J., Stallworth, L. & Malle, B. Social dominance orientation: a personality variable predicting social and political attitudes. J. Pers. Soc. Pychol. 67, 741–763 (1994).

  81. Hong, S.-M. & Faedda, S. Refinement of the Hong psychological reactance scale. Educ. Psychol. Meas. 56, 173–182 (1996).

    Article  Google Scholar 

  82. Brehm, J. W. A Theory of Psychological Reactance (Academic Press, 1966).

  83. Luhtanen, R. & Crocker, J. A collective self-esteem scale: self-evaluation of one’s social identity. Pers. Soc. Psychol. Bull. 18, 302–318 (1992).

    Article  Google Scholar 

  84. Kim, J. & Zauberman, G. Bias and ideology. Open Science Framework https://doi.org/10.17605/OSF.IO/SA35X (2023).

Download references

Acknowledgements

We thank R. Hauser for significant contributions during the early stages of this research project; C. Cusimano, B. K. Teeselink, S. Frederick, J. Dana, J. Kim, J. Wong and seminar participants at Yale University, the University of Chicago and Data Colada for insightful comments. The Yale School of Management and its Behavioral Lab provided funds to G.Z. for this research. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. The D’Amore-McKim School of Business, Northeastern University, Boston, MA, USA

    Jin Kim

  2. Department of Marketing, Yale School of Management, New Haven, CT, USA

    Jin Kim & Gal Zauberman

Authors
  1. Jin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gal Zauberman
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

G.Z. conceived the research question. G.Z. and J.K. developed the study designs. J.K. collected and analysed the data under the supervision of G.Z. J.K. wrote the initial draft, and G.Z. and J. K. wrote and edited the subsequent drafts. J.K. and G.Z. approved the final manuscript.

Corresponding authors

Correspondence to Jin Kim or Gal Zauberman.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks David Caicedo, Jarret Crawford, Amanda Kay Montoya and Cristian Rodriguez for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Sections including Figures and Tables.

Reporting Summary

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, J., Zauberman, G. The relationship between political ideology and judgements of bias in distributional outcomes. Nat Hum Behav 8, 228–242 (2024). https://doi.org/10.1038/s41562-023-01779-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-023-01779-3

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing