Abstract
Women (compared to men) and individuals from minority ethnic groups (compared to the majority group) face unfavourable labour market outcomes in many economies1,2, but the extent to which discrimination is responsible for these effects, and the channels through which they occur, remain unclear3,4. Although correspondence tests5—in which researchers send fictitious CVs that are identical except for the randomized minority trait to be tested (for example, names that are deemed to sound ‘Black’ versus those deemed to sound ‘white’)—are an increasingly popular method to quantify discrimination in hiring practices6,7, they can usually consider only a few applicant characteristics in select occupations at a particular point in time. To overcome these limitations, here we develop an approach to investigate hiring discrimination that combines tracking of the search behaviour of recruiters on employment websites and supervised machine learning to control for all relevant jobseeker characteristics that are visible to recruiters. We apply this methodology to the online recruitment platform of the Swiss public employment service and find that rates of contact by recruiters are 4–19% lower for individuals from immigrant and minority ethnic groups, depending on their country of origin, than for citizens from the majority group. Women experience a penalty of 7% in professions that are dominated by men, and the opposite pattern emerges for men in professions that are dominated by women. We find no evidence that recruiters spend less time evaluating the profiles of individuals from minority ethnic groups. Our methodology provides a widely applicable, non-intrusive and cost-efficient tool that researchers and policy-makers can use to continuously monitor hiring discrimination, to identify some of the drivers of discrimination and to inform approaches to counter it.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
To obtain access to the anonymized data, researchers have to sign a data sharing agreement with the KOF Micro Data Centre at ETH Zurich and the Labour Directorate of SECO. Source data are provided with this paper.
Code availability
Code to replicate all analyses presented here is available at the publicly accessible Harvard Dataverse: https://doi.org/10.7910/DVN/GGENFB.
References
Altonji, J. G. & Blank, R. M. Race and gender in the labor market. Handbook of Labor Economics 3, 3143–3259 (1999).
Blau, F. D. & Kahn, L. M. The gender wage gap: extent, trends, and explanations. J. Econ. Lit. 55, 789–865 (2017).
Neumark, D. Experimental research on labor market discrimination. J. Econ. Lit. 56, 799–866 (2018).
Bartoš, V., Bauer, M., Chytilová, J. & Matějka, F. Attention discrimination: theory and field experiments with monitoring information acquisition. Am. Econ. Rev. 106, 1437–1475 (2016).
Bertrand, M. & Mullainathan, S. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am. Econ. Rev. 94, 991–1013 (2004).
Bertrand, M. & Duflo, E. in Handbook of Economic Field Experiments Vol. 1 (eds Banerjee, A. V. & Duflo, E.) 309–393 (Elsevier, 2017).
Baert, S. in Audit Studies: Behind the Scenes with Theory, Method, and Nuance (ed. Gaddis, S. M.) 63–77 (Springer, 2018).
Kuhn, P. & Shen, K. Gender discrimination in job ads: evidence from China. Q. J. Econ. 128, 287–336 (2013).
Horton, J. J. & Tambe, P. Labor economists get their microscope: big data and labor market analysis. Big Data 3, 130–137 (2015).
Chan, J. & Wang, J. Hiring preferences in online labor markets: evidence of a female hiring bias. Manage. Sci. 64, 2973–2994 (2018).
Eriksson, S. & Lagerström, J. Detecting discrimination in the hiring process: evidence from an internet-based search channel. Empir. Econ. 43, 537–563 (2012).
Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58, 267–288 (1996).
Belloni, A., Chernozhukov, V. & Hansen, C. Inference on treatment effects after selection among high-dimensional controls. Rev. Econ. Stud. 81, 608–650 (2014).
Zschirnt, E. & Fibbi, R. Do Swiss Citizens of Immigrant Origin Face Hiring Discrimination in the Labour Market? Working paper 20 (Université de Neuchâtel, 2019).
Hainmueller, J. & Hangartner, D. Who gets a Swiss passport? A natural experiment in immigrant discrimination. Am. Polit. Sci. Rev. 107, 159–187 (2013).
Greenwald, A. G. & Banaji, M. R. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychol. Rev. 102, 4–27 (1995).
Chugh, D. Societal and managerial implications of implicit social cognition: why milliseconds matter. Soc. Justice Res. 17, 203–222 (2004).
Muraven, M. & Baumeister, R. F. Self-regulation and depletion of limited resources: does self-control resemble a muscle? Psychol. Bull. 126, 247–259 (2000).
Pocheptsova, A., Amir, O., Dhar, R. & Baumeister, R. F. Deciding without resources: resource depletion and choice in context. J. Mark. Res. 46, 344–355 (2009).
Danziger, S., Levav, J. & Avnaim-Pesso, L. Extraneous factors in judicial decisions. Proc. Natl Acad. Sci. USA 108, 6889–6892 (2011).
Olivetti, C. & Petrongolo, B. The evolution of gender gaps in industrialized countries. Annu. Rev. Econ. 8, 405–434 (2016).
Azmat, G. & Petrongolo, B. Gender and the labor market: what have we learned from field and lab experiments? Labour Econ. 30, 32–40 (2014).
Riach, P. A. & Rich, J. An experimental investigation of sexual discrimination in hiring in the English labor market. Adv. Econ. Anal. Policy 5, https://doi.org/10.2202/1538-0637.1416 (2006).
Eagly, A. H. & Steffen, V. J. Gender stereotypes stem from the distribution of women and men into social roles. J. Pers. Soc. Psychol. 46, 735–754 (1984).
Pager, D. & Shepherd, H. The sociology of discrimination: racial discrimination in employment, housing, credit, and consumer markets. Annu. Rev. Sociol. 34, 181–209 (2008).
Hagendoorn, L. Intergroup biases in multiple group systems: the perception of ethnic hierarchies. Eur. Rev. Soc. Psychol. 6, 199–228 (1995).
Phelps, E. S. The statistical theory of racism and sexism. Am. Econ. Rev. 62, 659–661 (1972).
Arrow, K. The Theory of Discrimination. Working paper 30A (Princeton University, 1971).
Becker, G. S. The Economics of Discrimination (Univ. Chicago Press, 1957).
Greenwald, A. G. & Krieger, L. H. Implicit bias: scientific foundations. Calif. Law Rev. 94, 945–967 (2006).
Bohnet, I., Van Geen, A. & Bazerman, M. When performance trumps gender bias: joint vs. separate evaluation. Manage. Sci. 62, 1225–1234 (2016).
Acknowledgements
We acknowledge funding from the Swiss National Science Foundation (grant no. 162620). We are grateful to SECO for collaborating on this project and sharing data, and we thank S. Thöni and M. Bannert for programming support.
Author information
Authors and Affiliations
Contributions
D.H., D.K. and M.S. designed the research; D.H., D.K. and M.S. performed the research and analysed the data; and D.H., D.K. and M.S. wrote the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature thanks Philipp Kircher and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
Extended Data Fig. 1 The search process on Job-Room.
The search process on the government-affiliated online recruitment platform of the Swiss public employment service. Left, stylized illustration of the search process; right, screenshots of every step. a–c, First the recruiter completes the search mask (a), typically by entering the occupation for which a vacancy exists and place of work, from which a ranked list of candidates who meet these criteria is generated (b). From this list, the recruiter selects the profiles that provide detailed productivity-related information akin to a standard CV about the jobseekers’ education, work experience and skills, as well as gender and markers of ethnicity (c). The bottom of the profile contains a button that reveals the contact information, typically a jobseeker’s phone number and email address, and the contact details of the employment office.
Extended Data Fig. 2 Validation of research design.
a, Percentage point effects and associated cluster-robust 95% confidence intervals of ethnicity on the likelihood that the contact button has been clicked conditional on profile visit (n = 254,975). The sample is restricted to non-registered recruiters who do not observe the nationality and name of jobseekers. Controlling for detailed language skills of jobseekers, these recruiters should not be able to infer candidates’ ethnicity. Standard errors are clustered at recruiter level. The null hypothesis that all ethnicity coefficients are jointly zero is not rejected (P = 0.93 (two-sided F-test)). b, Comparison of our baseline results that are based on the click on the contact button (in blue) with analogous regressions with outcome variables that are more direct signals of contact attempts by recruiters (n = 3,251,303). Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The dependent variable in the model ‘mail or print button’ is a binary indicator if a recruiter clicked on the hyperlink of the email address of the candidate or on the button ‘print candidate profile’ available on the profile page (n = 3,251,263 profile views). These click events happen in only 3.3% and 4.8% of profile visits, respectively. The dependent variable in the model ‘contact button (60 sec)’ is a binary indicator equal to one if a recruiter clicked on the contact button and subsequently stayed on the profile of the respective jobseeker for at least 60 s—a sign that the recruiter contacted the jobseeker immediately, possibly via phone or email. This happens in 27.3% of all contact attempts. The coefficients are normalized with the mean of the outcome variables to represent percent effects.
Extended Data Fig. 3 Discrimination by type of recruiter.
a, Effects of ethnicity on contact rate estimated separately for recruiters with 1 search (n = 62,081 profile views), 2–10 searches (n = 404,203 profile views), 11–50 searches (n = 546,441 profile views) and more than 50 searches (n = 2,238,538 profile views) in the study period. The coefficients are normalized with the mean contact rate of the respective category. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. b, Effects of ethnicity on the contact rate estimated separately for registered (n = 2,996,288 profile views) and non-registered (n = 254,975 profile views) recruiters of the platform. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The light red bars indicate minority ethnic groups that are not (or to a lesser extent) identifiable based on their language skills on Job-Room. Supplementary Table 11 provides an overview of the language skills of the different ethnic groups.
Extended Data Fig. 4 Effects by search duration within a session and for slow and fast deciders.
a, Effect of ethnicity on contact rate depending on the length of a session for immigrant jobseekers from Europe (in blue) and from outside of Europe (in red). Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression (n = 1,378,011 profile views). The estimates are derived from interactions between the indicators of ethnicity and indicators of the time elapsed since the start of the session. A session is defined as the time at which the recruiter logs into Job-Room until he or she logs off or closes the browser window. Longer sessions typically encompass several searches. As recruiters vary in their average session length, and these compositional differences can influence our results, the sample is restricted to sessions that take at least 30 min. The regressions control for session instead of search-fixed effects. Swiss jobseekers are used as the reference category. b, Effects of ethnicity on contact rate depending on the average decision time of a recruiter. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The sample is restricted to recruiters who conduct at least 6 searches and view at least 12 profiles. We use a random subset of one third of searches per recruiter to assign recruiters into one of four groups: fast deciders (lowest quartile), rather fast deciders (second quartile), rather slow deciders (third quartile) and slow deciders (highest quartile). The figure shows the ethnic penalties estimated separately for each group of recruiters using the test sample (the remaining two thirds of searches per recruiter). The average time that recruiters are looking at jobseekers’ profiles in the test sample are 5.7 s for fast deciders (n = 303,934 profile views), 9.9 s for rather fast deciders (n = 545,538 profile views), 15.4 s for rather slow deciders (n = 690,989 profile views) and 31.7 s for slow deciders (n = 364,018 profile views).
Extended Data Fig. 5 Additional gender analyses.
a, Effect of female gender on the probability of contact (in %) (n = 17,369,372 profiles). The circles show the (dis)advantage that women face compared to men in a given occupation, plotted against the share of women in that occupation. In contrast to Fig. 3 in the main text, the share of women per occupation is calculated from the Swiss earnings structure survey instead of the share of women in the result list. The circumference of the dots denotes the share of searches in each occupation. The solid black line indicates the weighted least squares regression of the estimated gender disparity against the share of female workers in each occupation. The dashed black lines show the associated 95% confidence intervals. The colour of the circles indicates the ISCO-1 level occupation classification. b, Relationship between occupational wages and hiring penalties (advantages) for female versus male jobseekers, grouped by two-digit ISCO occupation (n = 17,369,372 profiles). We estimate occupation-specific wages from the October 2016 wave of the Swiss Earnings Structure Survey. The circles show the (dis)advantage that women face compared to men in a given occupation, plotted against the log average wage in the occupation. The circumference of the dots denotes the share of searches in each occupation. The solid black line indicates the weighted least squares regression of the estimated gender disparity against the average log wage in each occupation. The dashed black lines show the associated 95% confidence intervals. The colour of the circles indicates the ISCO-1 level occupation classification.
Extended Data Fig. 6 Tests for statistical discrimination.
a, Effects of ethnicity on the contact rate estimated separately for jobseekers with low (n = 332,425 profile views), medium (n = 585,979 profile views) or high (n = 501,506 profile views) unobserved employability. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The construction of the index of unobserved employability exploit that we can observe the age, marital status and pre-unemployment wage of jobseekers. These characteristics are unobserved by recruiters on Job-Room. The construction involves several steps. First, we collapse the data at the jobseeker level and split the sample randomly in a training (50% of the searches) and a test sample (the remaining 50%). Second, we interact the three characteristics with nine occupation dummies (according to the Swiss SBN occupational classification). Third, we regress a dummy variable that a jobseeker leaves unemployment within the first four months after registering at the Swiss public employment service on these interactions, controlling for occupation-period and canton fixed effects. On the basis of these occupation-specific estimates, we predict the likelihood that jobseekers in the test sample leave unemployment within four months. We then assign each of these jobseekers to one of three equally sized groups (high, medium and low unobserved employability). The likelihood that a jobseeker with high unobserved employability leaves unemployment within 4 months is 37%, whereas it is 10% for jobseekers with low unobserved employability. The regressions are based on the test sample. b, Effects of ethnicity on the contact rate by length of the unstructured text field containing information on additional skills of jobseekers (see Supplementary Table 1 for examples). Dots with horizontal lines indicate point estimates of the interaction between length of skill field and ethnicity with cluster-robust 95% confidence intervals from OLS regression (n = 3,251,263 profile views). The reference categories (the hollow dots on the zero line) are Swiss jobseekers who report no additional skills, a skill field with 1 to 10 words and a skill field with more than 10 words, respectively.
Extended Data Fig. 7 Effect of a click on the contact button on the likelihood of leaving unemployment.
Effect and associated cluster-robust 95% confidence intervals of a click on the contact button on Job-Room on the likelihood that a jobseeker leaves unemployment, exploiting that we can link candidates to the unemployment register. The dependent variables are dummy variables equal to one if a jobseeker leaves unemployment within 30, 60, 90, 120 and 150 days, respectively, after a search. Each coefficient is derived from a separate regression (n = 12,823,811 profiles). The variable of interest is the contact attempt on Job-Room. The sample consists of all individuals that appeared within a result list of a search between March and September 2017. We drop the remaining periods because we observe unemployment exits only up to March 2018. We also exclude all registered unemployed individuals who reach their maximum benefit duration to make sure that exit out of unemployment represents a voluntary decision by the jobseeker. The regressions control for search-specific fixed effects and all variables visible to recruiters on Job-Room including unemployment duration at the time of the search and all relevant first-order interactions. The model contains 2,569 covariates that are predictive for leaving unemployment and the contact button click, selected by the Lasso-based post-double selection method. The regressions thus control for all factors that plausibly influence recruiters’ contact attempts on the platform. The coefficients are identified because certain individuals are contacted on the platform whereas observationally equivalent individuals who appear in the same search are not; for example, because of idiosyncrasies in recruiters’ decisions or because their candidate profile is not visited (for example, as they are ranked low in the result list). The average likelihood of leaving unemployment after 30 (60, 90, 120, 150) days is 17% (30%, 40%, 48%, 54%). The click on the contact button thus increases the likelihood to leave unemployment within 90 days after the search by 2.1% (0.821/40). Standard errors are clustered at recruiter level.
Supplementary information
Supplementary Information
The Supplementary Information contains supplementary methods and eleven supplementary tables that provide details on the data used in the analysis, the statistical model used to estimate ethnic and gender penalties, robustness and sensitivity analyses for these regressions, and extensions to our main regression results to draw out further implications.
Rights and permissions
About this article
Cite this article
Hangartner, D., Kopp, D. & Siegenthaler, M. Monitoring hiring discrimination through online recruitment platforms. Nature 589, 572–576 (2021). https://doi.org/10.1038/s41586-020-03136-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41586-020-03136-0
This article is cited by
-
Theory and rationale of interpretable all-in-one pattern discovery and disentanglement system
npj Digital Medicine (2023)
-
Responsible artificial intelligence in human resources management: a review of the empirical literature
AI and Ethics (2023)
-
A statistical approach to detect disparity prone features in a group fairness setting
AI and Ethics (2023)
-
COMPAS: zu einer wegweisenden Debatte über algorithmische Risikobeurteilung
Forensische Psychiatrie, Psychologie, Kriminologie (2022)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
