Validating a hazard-awareness assessment.

Test whether assessment scores predict observable behaviour.

The Hazard Awareness Test uses picture puzzles to assess hazard recognition. A shorter version, the 4pHAT, was intended to make the assessment more practical to administer. Earlier validation work had used concurrent methods; this study tested scores against behaviour measured later.

The practical question was relevant to employee selection: could performance on the assessment tell us something useful about a person’s ability to recognise hazards? The study examined that ability, rather than subsequent accident rates.

Build a study around a behavioural measure.

I conducted the research for my MSc in Applied Psychology at the University of Canterbury, supervised by Chris Burt with Joana Kuntz as associate supervisor. The project involved study design, participant testing, data preparation and statistical analysis, with technical support from the university and Talegent.

A central part of the method was an office-like “hazard lab” containing 16 simulated hazards. The task was piloted before the main study, with real danger removed and a consistent scoring approach for the hazards participants identified. The assessment tools were existing instruments; my work focused on evaluating their validity.

Separate the assessment from the follow-up task.

Seventy undergraduate psychology students took part, split evenly between the full HAT and the 4pHAT. They completed cognitive and personality measures as well as their assigned hazard-awareness assessment.

Approximately one week later, participants returned to identify and record hazards in the lab. Sixty-four completed this follow-up task. Acquaintance questionnaires provided a further source of ratings about safety-related behaviour, with responses available for 53 participants.

I examined correlations between assessment scores and the behavioural and rating measures, and explored whether hazard-awareness scores added predictive information beyond cognitive and personality measures.

Higher test scores were associated with more hazards identified.

The 4pHAT correlated with the number of hazards identified in the follow-up task (r = .53, n = 30). The full HAT also showed a positive relationship (r = .46, n = 34). Both correlations were statistically significant at p < .01.

These results supported criterion-related validity within the study setting. Chris Burt and I were subsequently listed as co-authors of a conference contribution reporting the short-test study at the 15th European Conference on Psychological Assessment in Brussels in July 2019.

Understand what the evidence can support.

The sample consisted of university students, and the task took place in a controlled environment without real danger. The findings therefore do not establish how the test would perform among employees facing workplace hazards, or whether using it in recruitment would reduce accidents.

Small samples also limited the regression analysis. Further research with occupational samples and real-world settings would be needed to assess wider applicability.

The lasting value for my assessment work is practical experience choosing a criterion, collecting evidence and interpreting validity alongside its limitations.

The research.

Validation of the 4pHAT — Jason Hunt, MSc dissertation, University of Canterbury, 2019 (PDF). Study design and participants: pp. 30–31; results: pp. 48–58; limitations: pp. 66–67.

A predictive criterion-related validation study of a short version of the Hazard Awareness Test — Christopher Burt and Jason Hunt, 15th European Conference on Psychological Assessment, 2019 (abstract, PDF p. 67).