Gender bias in the Online Job Search

From SI410
Jump to: navigation, search
There are many stages to the hiring process, and algorithms are involved in almost all of them. [1]

The advancement of technology and algorithms has led to companies recruiting and hiring potential employees through the use of big data and artificial intelligence, which can amplify gender bias in the online job search [2]. Gender bias refers to the “unfair difference” in the way that both men and women are treated [3]. In the context of the online job search, gender bias is defined as the advantage male job seekers have versus female job seekers. The use of algorithms in the online job search process can perpetuate existing biases which result in ethical implications regarding blocked job opportunities for females, and the disadvantage for female job seekers due to systematic gender roles.

Artificial Intelligence in the Recruitment Process

Companies utilize AI technology in their recruiting algorithms to recruit new employees more effectively. [4]. The introduction of AI into the recruiting process makes candidates 14% more likely to pass interviews and receive offers, 18% more likely to accept the job, but they are 12% less likely to inform recruiters of competing job offer during the negotiation process [5]. This technology is mainly used within four general types of recruitment activities: outreach, screening, assessment, and coordination [6].


Artificial intelligence helps companies find applicants that are the best fit. These potential applicants consist of active candidates (those who are deliberately searching for a new job) and passive candidates (those who are not actively searching but would show interest in the right opportunity). The AI uses data from websites like LinkedIn, Facebook, Twitter, etc. to match candidates to jobs. After enough training, the AI will have learned the most efficient way to word and present jobs to potential candidates. [6]


AI is also used for the resume screening process. AI can help companies reduce the time-to-hire. Hilton Hotels & Resorts implemented a screening tool that used AI and found an 88% decline in time-to-hire, or 42 days to 5 days. These AI screening tools can be more effective than humans because they can infer key terms from natural language. For example, instead of searching for the keyword, ‘persistence’, the AI can infer this term from other phrases or wording. [6]


AI assessments are used to narrow a candidate pool after the resume screening step. AI can be used in varying types of assessments. It can range from realistic chatbot conversations in situational judgement tests to making decisions based on an applicant’s responses to test questions [7]. These assessments can be used as an initial interview or to informally test candidates for desired traits, such as an individual’s risk propensity. [6]


It is beneficial for companies to make their hiring process as positive of an experience as possible because a candidate might not be the right fit today, but they could be perfect in a year. AI can facilitate this because it creates a more seamless, digital experience. Companies can use chatbots to update candidates on where they are in the process, fill in information gaps like the candidate’s potential start date, and answer candidate questions. [6] Also, a company’s openness with their usage of AI in the recruitment process increased the likelihood of a job application, as well as having a positive experience with the process [8].

Evidence of Bias

Process of Facebook's lookalike audience tool. [9]

Job Recruitment Algorithms

Job recruitment algorithms have been found to reinforce and perpetuate unconscious human gender bias [10]. Because job recruitment algorithms are trained on real-world data – and the real world is biased – the algorithms amplify this bias on a larger scale. Decision-making algorithms are “designed to mimic how a human would…choose a potential employee” and without careful consideration, algorithms can intensify bias in recruiting [11].

Utilizing real-world data to shape algorithms leads to algorithms producing biased outcomes. Algorithms make predictions by analyzing past data [2]. If the past data includes biased judgments, then the algorithm’s predictions will also be biased. Facebook’s tool called “lookalike audience” allows advertisers – in this case, employers – to input a “source audience” that will dictate who Facebook advertises jobs to, which is based on a person’s similarities with this “source audience” [2]. This tool is meant to help employers predict which users are most likely to apply for jobs. If an employer provides the lookalike tool with a dataset that does not include a lot of women, then Facebook will not advertise the job to women. Employers could use this tool to deliberately exclude certain groups, but employers could also be unaware of the bias of their “source audience” [2].

Job Recommendation Algorithms

Job recommendation algorithms within online platforms are built to find and reproduce patterns in user behavior, updating predictions or decisions as job seekers and employers interact [12]. If the system within the platform recognizes that the employer interacts with mostly men, then the algorithm will look for those characteristics in potential job applicants and replicate the pattern [12]. This pattern picked up by the algorithm can happen without specific instruction from the employer, which leads to biases going unnoticed.

Another problem that is encountered with job recommendation algorithms is the “lack of publicly available information.”[13] While companies can say that they are working to account for biases in the algorithm, the general public does not actually know how these problems are being tackled. However, the reason why this information is not public is due to the sensitivity of employee data that is used to train the algorithms. Because of this, the information that we can gather about practices in the industry is limited.[13]

Algorithms Extending Human Bias

Personal, human bias extends into algorithmic bias. A study conducted at the University of California, Santa Barbara found that people’s own underlying biases were bigger determinants of their likelihood to apply to jobs than any gendered job posting [14]. Underlying human biases need to be reduced to work towards gender neutrality in the job market [14]. Humans choose the data to train algorithms with, and the "choice to use certain data inputs over others can lead to discriminatory outcomes" [15]. Hiring algorithms can be an extension of "our opinions embedded in code" and further research highlights that algorithms reproduce existing societal, human bias [15]. The people constructing hiring algorithms are in the tech industry, which is not very diverse. This leads to algorithms that are trained on non-diverse data, which extends human gender bias into the online job market [15]. While creating algorithms, "biases creep in because human bias [influences] the algorithm" [16]. Humans build biased algorithms, so it is up to humans to notice the biases and fix them [16].

Ethical Implications

Algorithms Blocking Opportunities

Amazon's hiring algorithm was modeled after real-world data, where there are more men in the tech field. The difference in gender of employees is shown in the figure above [17].

Algorithms in the online job search do not outright reject job seekers. Instead, they block certain groups of job seekers from seeing opportunities they are qualified for; as Pauline Kim, a legal scholar, stated, “not informing people of a job opportunity is a highly effective barrier” to job seekers [12]. Qualified candidates cannot apply for a job if they have not been shown the opportunity.

Amazon’s algorithmic recruiting tool was trained with 10 years’ worth of resumes that were sent to Amazon; however, because technology is a male-dominated field, most of the resumes were from male applicants, leading the algorithm to downvote women [17]. This method of training taught the algorithm that men were preferred, therefore penalizing candidates who included the word “women’s” in their resume. For example, if the candidate listed an activity as “women’s team captain, ” their resume would be downgraded in the system [17]. Amazon has since scrapped this recruiting algorithm [18].

A test was also conducted on an ad-serving algorithm that displayed ads for jobs related to STEM. The algorithm was supposedly designed to display these ads to men and women equally and was tested in 191 different countries. The results showed that the ads were shown to around 20% more men than women. One of the explanations to the algorithm choosing to show the ads to more men than women comes from an economic standpoint. Online advertisers are constantly competing for users’ attention. A study has shown that on average, it is more expensive to get the attention of females over males when advertising online.[19]

Traditional Gender Roles Affect Outcomes

It has been studied that women can have a better keyword match on their resume, yet not be selected for a job if a man has more experience than them [11]. These hiring algorithms that are built and trained by humans do not take into account the time women must take off of work to have children or to take care of children [11]. The author of a study done by the University of Melbourne recounts that “women have less experience because they take time [off work] for caregiving, and that algorithm is going to bump men up and women down based on experience” [10]. Because women are more likely to experience a disruption in their career due to children, they will be viewed as a lesser candidate by the algorithm, even if they have more relevant experience than a male candidate [11]. Hiring algorithms do not take into account gender roles, which include women taking time off to give birth. This replicates and reinforces gender bias in the online hiring process [11].

Algorithmic Discrimination Towards Minorities

Graph showing how minorities are disproportionately shown blue-collar jobs from technology review

The financial success of digital advertising platforms is somewhat due to the exact targeting features they offer</ref>. Researchers and journalists found that advertisers can target and exclude particular groups of users seeing their ads for a product, service, or job position. Similarly, a slight amount of attention has been granted to the implications of the platform's ad delivery process, comprised of the platform's choices about which users see which ads.[20]

Developers meant for targeted advertising be efficient in matching advertisers to their customers. However, targeted advertising infrastructure can be leveraged and abused by malicious advertisers to reach people susceptible to false stories, stoke grievances efficiently, and incite social conflict. Since non-targeted and non-vulnerable people do not see the targeted ads, malicious ads are likely to go unreported and their effects undetected. [21]

A team led by Muhammad Ali and Piotr Sapiezynski at Northeastern University ran a series of otherwise identical ads with slight variations in the available budget, headline, text, or image. They found that those subtle tweaks had significant impacts on the audience reached by each ad, most notably when the ads were for jobs or real estate. Postings for preschool teachers and secretaries, for example, were shown to a higher fraction of women. In contrast, algorithms showed postings for janitors and taxi drivers to a higher proportion of minorities. Ads about homes for sale were also shown to more white users, while advertisers showed ads for rentals to more minorities.[22]

Bias is about more than just advertising job positions. In 2016, the rental platform Airbnb faced accusations that hosts on their site were discriminating by refusing reservations for black users. To address this, the company has said it will put new anti-discrimination clauses in place, change booking policies, and punish hosts who improperly reject potential guests. Ride-hailing companies have faced similar accusations of discrimination by those using their platforms. [23]

Algorithms carry the bias people have forward into technology, which disproportionately affects minorities and benefits those already in power. While the united states have laws that prevent employers from explicitly advertising job positions to a specific demographic, making it illegal to discriminate, computers can discriminate via targeting without detection due to their "black box" like infrastructure. As people become more educated on the topic of algorithms, awareness and regulation will follow.

Reducing Bias

Basic graphic of how researchers observed and evaluated gender bias of word embeddings in algorithms. [24]

Rethinking How Algorithms are Built

Vendors that build recruitment algorithms to target specific job seekers need to think beyond the minimum compliance requirements; they have to consider whether or not the algorithm they are building is leading to more fair hiring outcomes [12]. Additionally, the people stating that their algorithms will reduce bias in the hiring process have to build and test their algorithms while keeping that goal in mind, or else the technology will continue to undermine the online job search process [12]. Re-thinking algorithms and how to build them will begin to reduce bias in the job search, as many factors need to be considered.

One specific aspect of gender bias can be found in word embeddings. Word embeddings are binary, pre-trained models that assign words or phrases specific representations and meanings; these models have been found to reflect societal bias [25]. For example, words like "receptionist" and "nurse" are linked to "women" and "she" whereas words like "doctor" and "computer scientist" are linked to "men" and "he". In addition, there has been a “prevailing idealized concept of femininity” in the past.[26] The term “female” appears more than twice the amount of times the term “male” appears. This suggests that in most contexts, males are the default assumption over females when referring to someone.[26] Researchers have explored potential solutions to de-bias word embeddings by using methods such as building a genderless framework as well as teaching the algorithm gender-neutral word embeddings [24]. Such methods aim to minimize the difference between gendered words (i.e. male versus female) and maximize "the difference between the gender direction and other neutral dimensions” [24][27]. This allows algorithms to use or neglect the gender dimensions.

Balancing Humans and Algorithms

Implementing a balance between predictive algorithms and human insight is a promising solution for employers looking to use algorithms in their hiring process while reducing bias [15]. Using artificial intelligence and algorithms to parse through large amounts of data or applicants works well for processing. Balancing the processing by algorithms with the "human ability to recognize more intangible realities of what that data might mean" is the second step in the process of limiting algorithmic bias [15]. For a partnership between humans and algorithms to be successful within companies, they need to consciously and deliberately implement new practices [15]. Both algorithms and humans still need to be held accountable for reducing bias, and working together would encourage a good short-term solution to the phenomenon of gender bias in the online job search [15].

While aiming to strike a balance between humans and algorithms is a promising solution, there is an important question to answer. What is considered “fair” when trying to find qualified candidates for a job. Most conversations have been geared towards “treating similar individuals similarly.”[28] However, a study has shown that a “model cannot conform to more than a few group fairness metrics at the same time.”[28]

Unintended Consequences of Bias Reduction

It has been suggested that algorithms de-biased in terms of gender could still produce the same biased outcome. Algorithms may still use online proxies in their scoring process to produce discriminatory results as these proxies serve as stand-ins for protected groups, like gender [29][30]. For example, while algorithms may be de-biased on the terms of gender specifically, the algorithm may use stand-in factors such as height or weight as a proxy to determine a candidate's gender [29]. This leaves room for the algorithm to produce bias results based on a candidate's gender.

Although there is a statistical process that is known to eliminate proxy discrimination, the process requires the algorithmic model to include "training data information on legally prohibited characteristics” [31]. Even if such legally prohibited information is obtained, characteristics would then be measured on their predictive power of the target variable; this could result in unintended amplification of the initial proxy discrimination [31]. For example, if height was measured as a highly predictive characteristic and height was used as a proxy for gender, then the algorithm would intentionally discriminate on the predictive basis of height thus unintentionally discriminating against gender.


  1. Bogen, M. & Rieke, A. (2018). Help Wanted: An Examination of Hiring Algorithms, Equity, and Bias. Upturn.,%20Equity%20and%20Bias.pdf
  2. 2.0 2.1 2.2 2.3 Kim, P. T. (2019). Big data and artificial intelligence: New challenges for workplace equality. University of Louisville Law Review, 57(2), 313-328.
  3. Cambridge Dictionary.(n.d.). Gender Bias. In Cambridge English Dictionary.
  4. (2021, February 26). Ai for recruiting: A definitive guide for hr professionals. Ideal.,repetitive%2C%20high%2Dvolume%20tasks.
  5. Cowgill, B. (2018). Bias and productivity in humans and algorithms: Theory and evidence from resume screening. Columbia Business School, Columbia University, 29.
  6. 6.0 6.1 6.2 6.3 6.4 Black, J., & Esch, P. (2019, December 31). Ai-enabled recruiting: What is it and how should a manager use it?
  7. (n.d.). Artificial intelligence (ai) in assessment. Aon.
  8. Esch, P., Black, J., & Ferolie, J. (2018, September 17). Marketing ai recruitment: The next phase in job application and selection.
  9. Facebook. (n.d.).
  10. 10.0 10.1 Hanrahan, C. (2020, December 2. Job recruitment algorithms can amplify unconscious bias favoring men, new research finds. The ABC News.
  11. 11.0 11.1 11.2 11.3 11.4 Cheong, M., et al. (n.d.). Ethical Implications of AI Bias as a Result of Workforce Gender Imbalance. The University of Melbourne.,-UniBank.pdf
  12. 12.0 12.1 12.2 12.3 12.4 Bogen, M. (2019, May 6). All the Ways Hiring Algorithms Can Introduce Bias. Harvard Business Review.
  13. 13.0 13.1 Raghavan M., Barocas S., Kleinberg J., Levy K. (6 December 2019). "Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices" Retrieved on 9 April 2021
  14. 14.0 14.1 Tang, S., et al. (2017). Gender Bias in the Job Market: A Longitudinal Analysis. ACM on the Human-Computer Interaction. .
  15. 15.0 15.1 15.2 15.3 15.4 15.5 15.6 Raub, M. (2018). Bots, Bias and Big Data: Artificial Intelligence, Algorithmic Bias and Disparate Impact Liability in Hiring Practices. Arkansas Law Review, 71(2), 529-570
  16. 16.0 16.1 Rosenbaum, E. (2018, May 30). Silicon Valley is stumped: A.I. cannot always remove bias from hiring. Consumer News and Business Channel.
  17. 17.0 17.1 17.2 Dastin, J. (2018, October 10). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters.
  18. Amazon scrapped 'sexist AI' tool. (2018, October 10). British Broadcasting Corporation.
  19. Lambrecht A., Tucker C. (2019). "Algorithmic Bias? An Empirical Study of Apparent GenderBased Discrimination in the Display of STEM Career Ads" Retrieved on 9 April 2021.
  20. Ali, Muhammad. "Discrimination through Optimization: How Facebook's Ad Delivery Can Lead to Skewed Outcomes." ArXiv.Org, 3 Apr. 2019,
  21. Ribeiro, Filipe N., et al. "On microtargeting socially divisive ads: A case study of russia-linked ad campaigns on facebook." Proceedings of the Conference on Fairness, Accountability, and Transparency. 2019.
  22. Hao, Karen. "Facebook's Ad-Serving Algorithm Discriminates by Gender and Race." MIT Technology Review, 2 Apr. 2020,
  23. White, Gillian. "The Trouble With Discrimination In Online Advertising." The Atlantic, 7 Mar. 2017,
  24. 24.0 24.1 24.2 Sun, T., et al. (2019, June 21). Mitigating Gender Bias in Natural Language Processing: Literature Review. Cornell University.
  25. GeeksforGeeks. (2020, October 14). Word Embeddings in NLP.
  26. 26.0 26.1 Leavy S., Meaney G., Wade K., Greene D. (12 July 2020) "Mitigating Gender Bias in Machine Learning Data Sets" Retrieved on 9 April 2021
  27. Bolukbasi, T., et al. (2016). Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. Advances in Neural Information Processing Systems 29 (NIPS 2016)
  28. 28.0 28.1 Silberg J., Manyika J. (6 June 2019). "Tackling bias in artificial intelligence (and in humans)" Retrieved on 9 April 2021
  29. 29.0 29.1 Larson, J., Mattu, S., & Angwin, J. (2015, August 31). Unintended Consequences of Geographic Targeting. Technology Science.
  30. Zarsky, T. Z. (2014). Understanding Discrimination in the Scored Society. Washington Law Review, 89(4), 1375-1412.
  31. 31.0 31.1 Prince, A. E.R. & Schwarcz, D. (2020). Proxy Discrimination in the Age of Artificial Intelligence and Big Data. Iowa Law Review, 105(3).