When used correctly, numerical scales are an effective way to rate a candidate’s interview or other assessment performance and quickly calculate their standing against fellow candidates.
When used correctly, numerical scales are an effective way to rate a candidate’s interview or other assessment performance and quickly calculate their standing against fellow candidates.
Whether you use a 1-5, 1-7 or 1-10 rating scale, the task of determining the highest ranked candidate becomes that much easier if you are able to add or average each candidate’s scores to obtain an overall picture of their performance.
However, a problem that sometimes occurs when using numerical rating scales is that a candidate scoring very well on less important criteria and poorly on essential criteria could end up with a higher total score and therefore ranked above candidates assessed as average performers across all selection criteria.
This type of problem signifies that each selection criterion should have a different value depending on its importance in the execution of a job. It is the process of “weighting” selection criteria to reflect their relative value that allows rating scores to be added and averaged without distorting the final outcome.
Unfortunately most weighting decisions tend to be a matter of guesswork, which may ultimately detract from the accuracy and defensibility of short-listing and selection decisions.
What to consider when weighting selection criteria
Research reveals four key areas for determining the value of each selection criterion. These include:
Relative Importance – Decision-makers familiar with the inputs and outputs of a role, the environment in which it operates and its current context should be able to determine which selection criteria contribute most to successful job performance. The highest priority or weight would naturally go to the most important criteria.
Consequence of Failure – Decision-makers may also determine how damaging failure from incompetent or delinquent behaviour would be in relation to each selection criterion. The consequences of failure may take into account such things as injury to people, damage to property or equipment, disruption to colleagues or other work areas, customer dissatisfaction, diminished brand perception, legal liability, and so on.
Time Spent in Performance - The time spent engaged in the activities or demonstrating the behaviours associated with a particular selection criterion will also indicate its value. Of course, time spent in performance should be calculated as a function of a selection criterion’s Relative Importance and Consequence of Failure so that the time spent on critical tasks or demonstrating important behaviours is weighted more heavily.
Time to Achieve Competence - For some selection criteria there will be a period of orientation and gaining of job specific knowledge, skill and experience that contributes to a person’s eventual effectiveness in a role (e.g. familiarisation with culture, customers, products or services, technology, procedures, etc.). Due to the organisation's greater investment and increased risk, a selection criterion should be considered more critical the longer it takes a person to become competent on-the-job. Once again, this figure should be calculated as a function of a selection criterion’s Relative Importance and Consequence of Failure so that time spent developing competence in important areas is given greater weight.
How to calculate selection criteria weights
Fenchurch has developed an online Selection Criteria Weight Calculator to simplify the task of assigning importance to selection criteria. Readers are encouraged to contact the author at [email protected] or visit www.fenchurch.com.au/contact to obtain free access to the calculator.
Whether you use a 1-5, 1-7 or 1-10 rating scale, the task of determining the highest ranked candidate becomes that much easier if you are able to add or average each candidate’s scores to obtain an overall picture of their performance.
However, a problem that sometimes occurs when using numerical rating scales is that a candidate scoring very well on less important criteria and poorly on essential criteria could end up with a higher total score and therefore ranked above candidates assessed as average performers across all selection criteria.
This type of problem signifies that each selection criterion should have a different value depending on its importance in the execution of a job. It is the process of “weighting” selection criteria to reflect their relative value that allows rating scores to be added and averaged without distorting the final outcome.
Unfortunately most weighting decisions tend to be a matter of guesswork, which may ultimately detract from the accuracy and defensibility of short-listing and selection decisions.
What to consider when weighting selection criteria
Research reveals four key areas for determining the value of each selection criterion. These include:
Relative Importance – Decision-makers familiar with the inputs and outputs of a role, the environment in which it operates and its current context should be able to determine which selection criteria contribute most to successful job performance. The highest priority or weight would naturally go to the most important criteria.
Consequence of Failure – Decision-makers may also determine how damaging failure from incompetent or delinquent behaviour would be in relation to each selection criterion. The consequences of failure may take into account such things as injury to people, damage to property or equipment, disruption to colleagues or other work areas, customer dissatisfaction, diminished brand perception, legal liability, and so on.
Time Spent in Performance - The time spent engaged in the activities or demonstrating the behaviours associated with a particular selection criterion will also indicate its value. Of course, time spent in performance should be calculated as a function of a selection criterion’s Relative Importance and Consequence of Failure so that the time spent on critical tasks or demonstrating important behaviours is weighted more heavily.
Time to Achieve Competence - For some selection criteria there will be a period of orientation and gaining of job specific knowledge, skill and experience that contributes to a person’s eventual effectiveness in a role (e.g. familiarisation with culture, customers, products or services, technology, procedures, etc.). Due to the organisation's greater investment and increased risk, a selection criterion should be considered more critical the longer it takes a person to become competent on-the-job. Once again, this figure should be calculated as a function of a selection criterion’s Relative Importance and Consequence of Failure so that time spent developing competence in important areas is given greater weight.
How to calculate selection criteria weights
Fenchurch has developed an online Selection Criteria Weight Calculator to simplify the task of assigning importance to selection criteria. Readers are encouraged to contact the author at [email protected] or visit www.fenchurch.com.au/contact to obtain free access to the calculator.
