Is your AI hiring system making you racist?

When an AI hiring tool builds in racial bias, human decision-makers often follow its lead rather than correcting it. 

That’s what a recent study by the University of Washington found — a pattern that researchers described as “participants mirrored the AI.”  

The experiment shows that bias isn’t just embedded in code; it seeps into the judgment of people who treat machine output as more objective than their own instincts. 

For Canadian organizations racing to automate high-volume recruiting, it’s a warning that plugging AI into unchanged hiring habits without proper analysis or safeguards can quietly scale up discrimination inside processes they think they are supervising, according to Wendy Cukier, professor of entrepreneurship and innovation at the Ted Rogers School of Management at Toronto Metropolitan University (TMU). 

However, automation is surging anyway. A recent report by Resume.org found that 57 per cent of organizations are using AI in their hiring process and one-in-three expect AI to run their entire hiring process by 2026. In addition, three-in-four organizations are already comfortable letting AI reject candidates without human oversight, signalling a deep trust in such systems.  

The risk of encoding hiring bias 

Cukier — who’s also the founder of the Diversity Institute at TMU — has spent years studying how supposedly objective hiring screens can push qualified people aside. She points to resumé experiments that altered nothing but the name at the top as an example. 

Almost invariably, an ethnic-sounding name would get fewer callbacks than a Western European name, she says, "and that was before AI, it was just human sorting mechanisms... So, the fact that AI processes replicate the best and the worst out of human decision-making shouldn't come as a surprise to anybody.” 

Cukier believes those patterns don’t vanish when organizations deploy AI to rank resumés, as they’re baked into the data from which the systems learn. “The real technical response is ‘garbage in, garbage out,’” she says. “AI systems are trained, and the data that they are trained with shapes the decisions that they make.” 

The University of Washington study underlines how fragile the “human in the loop” promise can be when people treat an AI recommendation as the default, then move only slightly around its suggestions instead of interrogating it, adds Cukier. 

Human at the centre or human on autopilot? 

In policy statements, AI is often framed as decision support for HR, yet Cukier sees hiring managers sliding into reliance that’s anything but cautious. “What the findings suggest is that increasingly hiring managers are using AI as an assistant, but they are taking its advice without applying another layer of scrutiny,” she says.  

She argues that real safeguards require more than a token reviewer rubber-stamping output.

“One of the things that we really emphasize when we talk about responsible AI use is this notion of the human at the center,” she says. “If you don't have human judgment factored in to check AI and to challenge some of the assumptions, you're just going to reinforce and replicate bias on a much larger scale than and in a much shorter time than than previously was the case.”  

That work starts with admitting how quickly bias surfaces, even in people who have spent their careers trying to dismantle it. Cukier notes that when she took the Harvard implicit bias test, she “associates men with technology and women with arts and science when I’m pushed to make decisions very quickly” — a reminder that no one gets a free pass on reflexive and biased responses.  

She points to strict laws in some jurisdictions such as the U.K. and now Ontario, where organizations have to disclose the use of AI in hiring, as a start. However, she says the onus is on the organizations using AI to take steps to address the risk of biased hiring.  

“Some of the tried-and-true processes in HR that have been most effective are looking carefully at the job design, making sure you don't have qualifications that aren’t actually tied to the competencies needed in the job, being attentive to the possibility of what I call ‘old boy bias,’” says Cukier. “Then making sure that the recommendations [from AI] are recommendations, not decisions.” 

She also stresses the importance of being aware of tricky spots in recruiting that AI could replicate, such as women who have taken maternity leave and people with disabilities who have gaps in their resumes and get passed over because of it. 

Liability moving faster than training, leadership 

Regulators and courts are increasingly interested in how automated tools shape access to work, especially when protected groups are disproportionately screened out, and HR leaders who rely on vendors’ assurances without demanding evidence are exposing their organizations to legal risk that will be hard to defend, says Cukier. 

She suggests that organizations should treat AI hiring systems as high-risk infrastructure, not simple productivity hacks, and to push for transparency on training data and performance, insist on independent audits and stress-test outputs for disparate impact on different demographic groups, before deploying those tools fully. 

Cukier also expresses surprise at the extent to which organizations don’t have good processes in place to address embedded bias in their systems. 

“It’s an ongoing process, whether it's about AI processes or looking at ways in which bias gets embedded in job descriptions, in selection processes, in promotion, in outreach,” she says. “We really have to look at the pathways to employment, and at every stage at which bias can creep in, whether it's AI-enabled or human-enabled.” 

Using AI to expose inequity, not hide it 

Even with the risks, Cukier doesn’t dismiss the utility of using AI tools for hiring and other tasks. 

“We've actually seen AI used to strip out bias — I think the results of this are mixed, but they tried in the U.K. to use AI to anonymize applicants' information,” she says. “That can also be a strategy, but at the end of the day, it's the human in the loop and transparency around the decisions that’s ensuring that the data being used to train the machine is as unbiased as possible.” 

The question facing Canadian HR leaders is whether they will let AI hiring tools quietly encode yesterday’s bias into tomorrow’s workforce, or whether they will use the same technology to address where the system is failing people and force the changes that have been promised. 

“Canada’s legislation hasn’t changed, and its Charter of Rights and Freedoms hasn’t changed,” says Cukier. “And I happen to be someone who thinks that AI is inevitable and it could make big differences in terms of productivity improvements — but if you mess it up, the costs are massive, not just from a financial or legal perspective, but also a reputational perspective.” 

“I believe AI is a tool — it's no better or worse than the information that's put in and the people who are using it.”