Unravelling psychometrics: Testing in the workplace

Chris Golis, executive chairman of Nanyang Ventures has used a range of tests since 1996 to evaluate the CEOs/entrepreneurs of potential investees. Nanyang Ventures is an Australian venture capital fund manager that provides equity capital and guidance to small to medium-sized businesses in return for a minority ownership stake. It runs a diagnostic test on the businesses on offer and then tests the CEOs that are running them.

“Over that time, we have invested in 30 companies. In this industry, you would typically expect half to fail. We’ve only had two failures and on both those occasions, we discounted the test results and trusted our intuition. We thought, ‘They can’t be that dumb.’ But they were that dumb. Our intuition was wrong,” says Golis. Nanyang requires the IQ of its CEOs to be in the top 10 per cent of the population.

Michelle Leung is head of people capital for ANZ rural banking. Psychometric testing was introduced in that business unit three years ago to assist in the recruitment of senior managers and executives, following a large restructure. Fifty people have undergone testing and none of the successful applicants has failed in their jobs. Two have been promoted to other roles and the rest are still working well in the jobs for which they were recruited. “Our business unit has a very rigorous recruitment and selection process and has been highly successful. People are very keen to work in rural banking at ANZ,” says Leung.

Paradigm shift

Kevin Chandler, director of Chandler & Macleod, has been administering psychological tests since 1969 and believes there’s been a paradigm shift in testing in the past few years. “In the old days, we used to test people and give the client a generalised description of the person applying for a job. People used to look at their results and were able to say: ‘That’s a pretty accurate description of me.’ But now, competencies are front and centre, so testing has stepped up a level and people are now being tested on the core competencies required for a job,” he says.

Nanyang Venture’s Chris Golis agrees. At the outset, they tested against competencies they thought would make a good leader of a growing business – leadership, vision and achievement drive. These were well accepted traits in leaders at the time, described by Golis as “arrogant and opinionated”. But they soon discovered that this type of leader probably worked best at the big end of town. They worked with their provider and have now come up with the core competencies that they believe are critical in growing a small to medium-sized business.

“They have to be smart and they have to have a commercial focus, so we test IQ and numerical reasoning. Some people think you can hire a good CFO to help with understanding the financial side of the business, but we’ve found that if the CEO doesn’t have it, a hired CFO cannot compensate for that,” says Golis. Nanyang Ventures also has the CEOs tested for integrity and team building, the two other critical success factors they have identified.

At ANZ rural banking, Leung says that the testing is done for a specific role and she would never hand over the results if that person were applying for a different role in the organisation. They would have to be re-tested against the competencies for that job. “However, the data we get [from the testing] is quite rich, so we do use it as a basis for that individual’s staff development plan.

In a further development of the use of testing, Nanyang’s provider also supplies them with a ‘management guide’ about the applicant. Golis says it describes the best way to gain this individual’s cooperation. For example, they might prefer using a whiteboard to a spreadsheet. “You have to remember, we are often in competition with other venture capital companies [to become a minor shareholder in the business], so if we have more empathy with them than other investors, they are more likely to choose us,” he says.

More than just testing

“You can’t use testing on its own – it must be part of a broader process,” Leung stresses. Rural banking also looks at the person’s work experience, does thorough reference checking and uses structured interviews. For the executive level positions, the business unit has also used simulation testing to assess strategic thinking ability.

Nanyang Ventures uses a reference checking company to do reference checks on the CEO investees. The results of the 10 reference checks usually corroborate what the psychologist’s report says, according to Golis. All good tests have built-in lie detector questions, to catch out people who try to answer according to what they think an employer might want, rather than just truthfully. Golis says that Nanyang has rejected people because they have lied on the tests. Integrity in the CEO is vital for Nanyang’s investment to proceed.

What does a good test look like?

Sarah Kearney is managing director of SHL Australia, a provider of a range of psychometric tests. Kearney says that a company should find out what the reliability and validity coefficients are for any test and that it’s the test publisher’s responsibility to provide this information. Reliability refers to the repeatability of a test – that is, if someone sits it today, will the results be about the same if they sit it on many other days. Kearney says this correlation should be 0.7 or above (1.0 is perfect). Validity refers to how well an attribute predicts performance. That should be at least 0.4 or above. “The single best predictors are cognitive ability tests such as verbal and numerical reasoning,” says Kearney. “Any tool is only valid when it is used for what is was designed for.”

The other question to ask is what norm group it’s used for. Although one of the requirements of any test is that it passes an anti-discrimination test, Kearney asserts that all tests are actually designed to discriminate – on whether people can do a particular job or not. A test should not be discriminatory against a particular race, age or colour, but it should discriminate between someone who has particular traits that match job competencies and someone who doesn’t.

How much does it cost?

Well, how long is a piece of string? It depends on what you want to have done, how much you’ve already prepared by yourself and how many times you want to do it. Some companies have individuals trained in administration and interpretation of tests. If you’ve got someone like that on your staff, a battery of tests may only cost $200–$300 per person. If you need assistance in developing competency models for a range of roles and have just one person put through some very tailored testing for a whole day, you might pay up to $5,000–$7,000. Typically though, testing costs companies $800–$2,000 per person.

Leung believes it’s easy to justify the cost of testing when you consider the cost of replacing a senior manager is around twice their annual salary. Golis sees the cost as a vital part in assessing the ability of the CEO who will be receiving millions of their dollars in venture capital. Given both companies’ success with psychometric testing, neither would ever try building a business case against testing.

A word about mediums

There has been a recent explosion in the availability of online testing from suppliers all round the world. Both Chandler and Kearney agree that online testing should be supervised, either in the client’s premises or the test provider’s.

“You never know how many people are sitting around a home computer answering the questions,” says Chandler. Kearney adds that the testing medium should be the best one to suit a particular business. “If you are testing a group of factory workers who are not all computer literate, it may be unfair to have online testing, if you are searching for traits that a good factory supervisor needs and one of them is not the ability to use a computer,” concludes Kearney.