Best practice is crystal clear

INNOVATIONS IN assessment technology have created more opportunities for richer candidate insights. However, these new technologies are not without risk, notes Dr Matthew Neale, chief psychology officer at Revelian.

“How do you make sure you’re taking advantage of new and technologically advanced hiring methods, while still providing a fair, equitable and reliable recruitment process?” says Neale.

Glass box versus black box
The key to answering these questions, Neale explains, turns on the difference between black box and glass box algorithms. Black box algorithms use forms of AI and machine learning that have free rein to combine and recombine the data in ever more complex ways to improve the prediction of an outcome, such as turnover or hiring success.

“We call these approaches ‘black box’ because the algorithms they produce are so complex that even the assessment designers can’t explain how they work,” says Neale.

While this approach can be useful, it also carries significant risk: the algorithms can’t be explained, and the outcomes they produce can’t be defended. This can have legal implications if your organisation’s hiring practices are ever challenged in court.

“How can you defend a hiring decision if you don’t know what led to that decision being made?” says Neale.

It’s also difficult to know if the algorithm has ‘inherited’ biases from the data it was trained on – gender, education or previous experience, for example. Finally, these algorithms can pick up on transitory flukes in the data to produce prediction results that are not sustainable or that decay over time. Without knowing how the algorithms work it’s not possible to gauge the ‘shelf life’ of the algorithm.

“The key thing that defines a glass box approach is that differences between candidates’ scores or outcomes can be explained” Dr Matthew Neale, chief psychology officer, Revelian

“All of these factors ultimately decrease trust in the algorithm and the utility of the assessments they are based on,” says Neale. “For the employer, the implications of black box assessments can undermine their efforts to build diverse and effective teams.”

An alternative path – one used at Revelian – is to follow a glass box approach. Geared at predicting outcomes and providing key insights into applicants, this approach still harnesses the power of machine learning and algorithmic insight but retains the capacityfor humans to understand and explain how outcomes are predicted. The algorithms summarise key features in the data, rather than amplifying its complexity. This means that glass box approaches can be used to predict an outcome and help your organisation understand why some applicants are better than others.

“The key thing, though, that defines a glass box approach is that differences between candidates’ scores or outcomes can be explained by the people who developed or use the assessment – they’re much more easily defendable,” says Neale. “Day-to-day, it also shows candidates that you’ve used appropriate tools to give everyone an equal opportunity.”

Five traits of quality psychometric test providers
Ethical providers will equip you with the information you need to confidently harness psychometric insights. This includes details of validation studies undertaken to ensure the assessments measure what they should, without unintentional bias. 

“If you’re considering using psychometric tools to enhance and streamline your recruitment process, there are a few points worth keeping in mind as you look for a reputable provider,” says Neale.

1. Measure the right traits

Assessing the right mix of traits and abilities for roles in your organisation is critical; it can only be done based on a deep understanding of what makes you and your business unique.

What to look for: Best practice providers design their tests to produce solid, evidencebased insights. They have advisers with psychometric qualifi cations who can suggest the right selection of tests for you.

2. Take control of your assessments

To harness the power of psychometric insights, it’s critical to understand how the mechanics of tests inform your hiring decisions. Technical support should also be available to you and your candidates whenever you need it so you can get the most out of assessments.

What to look for: Quality providers support and train you to use easy-tounderstand reports and measures so you can confidently back your hiring decisions with reliable tools.

3. Support your legal and ethical integrity

AI streamlines recruitment processes, but speed doesn’t always support fairness or accuracy. A poorly designed test with algorithmic bias will skew insights and expose you to ethical, legal and reputational consequences.

What to look for: A reliable test produces consistent insights to inform your fair hiring decisions.

4. Create positive candidate experiences

Powerful candidate experiences help your organisation attract and retain the right people. They bring together the best recruitment strategies, processes and tools. Any compromise in quality detracts from your candidates’ experiences.

What to look for: Leading providers develop innovative and user-friendly tests that boost your employer brand and hiring practices. They should also uphold stringent data security, privacy and quality requirements to protect and regulate data use.

5. Trust your investment

The psychometric tools you use need to be quantifiable and should contribute to your business-critical outcomes. The best assessments are developed by teams of experienced software engineers and psychologists.

Revelian is an Australian-based company specialising in data-driven human behaviour insights. We support and partner with our clients so they can confidently make effective hiring and development decisions.
Our assessments are interactive, reliable and flexible. We’re trusted by top employers around the world to deliver powerful HR insights through cutting-edge psychometric assessments. For more information, visit www.revelian.com.