Not all assessments are made equal: Here’s how to spot the good ones.

From fun to fundamental 

Assessments are used in various contexts, for various reasons, from a fun activity in a magazine, to a potential life-changing diagnosis or decision. From the outside assessments all look similar … basic structured questions, often with a limited number of pre-set answers.  

The more serious the context, the more you need to make sure that the assessment is in fact a good one. You may not want to make a career decision on whether you are a Hufflepuff or a Gryffindor! 

Ok, so what makes a good assessment? 

In South Africa, the Employment Equity Act prohibits the subjection of employees to psychological testing unless the test or assessment being used (1) is reliable and valid, (2) is fair, and (3) is not biased against anyone. This means that the test should function as intended, accurately measure what it was designed to measure, and no person or group should have an advantage over another when completing the test for selection or any other purpose.   

Is it valid, reliable, fair, and unbiased? 

Ask for a technical manual.

How do you know if the above-mentioned criteria have been met? The technical manual typically contains information related to a test’s psychometric properties. Therefore, when using ANY assessment, you should request the technical manual and evaluate the test’s psychometric properties, as it is ultimately your responsibility to ensure that the test is fit for the context in which it is being used. These manuals should be available to you for free, no NDA required.

It's on ‘the list’ it should be OK. Right? Wrong! 

The Health Professions Council of South Africa (HPCSA) has a list of assessments that indicates whether the assessment is psychological or not. The intention of this is to determine who may use the assessment. It does not indicate whether the assessment is any good. To this end, the Assessment Standards South Africa (ASSA) was formed to review whether assessments meet the minimum requirements to be considered in high-stakes scenarios. This is a voluntary process and JVR has already submitted a number of our assessments (with more in the pipeline). 

Who are you compared to? (and so, what?)

Many assessments are designed to compare you against others – to see where you fall on a normal curve. This sample of ‘others’ is called a norm. A good assessment should have appropriate norms for its target audience, sufficient research to ensure that the norm sample is similar to the population under scrutiny, clear guidelines on how to administer, score, and interpret the test, a strong scientifically grounded and tested framework, and there should be clarity regarding the test’s purpose and its most suitable areas of application (e.g., is it appropriate for research, development, selection, other?). Ultimately, for an assessment to be good, it needs to meet the above criteria AND predict real-world phenomena, i.e. job performance, resilience, wellbeing, etc ...

Working with an assessment professional?  

Globally it is expected that if you are working with assessments that you should have appropriate training and qualifications to work with them ethically. 

In South Africa, there are legal requirements on certain categories of assessments ensuring that only appropriate registered individuals use certain types of assessments. 

Ask these questions: 

• Does the assessment have a technical manual available that you’ve reviewed?  

• When last has the technical manual been updated or reviewed? (Should be every five years) 

• Has the assessment been classified by the HPCSA as a psychological test?  

• Has the quality of the assessment been reviewed by the Assessment Standards of South Africa? (This is not compulsory in South Africa but highly recommended) 

• Are the norms being used appropriate for the intended audience in my organisation?  

• Has the assessment been proven to predict ‘performance’ not only potential (Has there been criterion studies done?) 

The JVR Africa Group has been working with assessments – developing, adapting, and validating them for the African context, since 1993.  Do you have a specific question? Chances are, we have a (scientifically-sound, reliable, and valid) assessment for that …