Evaluating the effectiveness of mandatory working-at-heights training standards

In 2015, a training standard for construction workers using fall protection equipment became mandatory in the province of Ontario. By 2017, 418,000 workers had been trained to the new standard. This study sets out to answer two questions: 1) To what extent does the working-at-heights training affect safety practices at the worksite? 2) Has the incidence of fall-from-heights injuries changed with the introduction of the training standard?

To answer the two questions, the research team recruited about 630 learners who received training from September to November 2017. They were asked to complete a questionnaire at three intervals—i.e. one week, four weeks and seven weeks—after training. About 430 of the participating learners also gave the research team permission to see the results of their safety knowledge tests, which were given by instructors before and after the training.

In addition, the team analyzed compensation claims data held by the Workplace Safety and Insurance Board, the provider of workers’ compensation insurance in the province. The analysis focused on three categories of claims in the construction sector, namely: 1) targeted falls (falls from heights due to events targeted by the training standard, such as falls from ladders and roofs); 2) untargeted falls (falls at the same level and falls from heights due to events not targeted by the mandatory training, such as falls down stairs); and 3) other acute traumatic injuries.

Knowledge gain: A large majority of the 630 learners reported gaining “a lot” or “some” new knowledge as a result of the training (52 per cent and 34 per cent respectively). A strong majority perceived the new information as ‘‘useful” or ‘‘very useful” (38 per cent and 51 per cent respectively). Respondents tended to report increasing their confidence in using seven working-at-heights practices (ranging from 60 per cent to 87 per cent across the items). Most intended to change their own safety practices as a result of the training (36 per cent ‘‘likely” and 47 per cent ‘‘very likely”).

Moreover, among the learners who consented to share the results of their knowledge tests, a large improvement in knowledge was indicated by a 40 per cent increase in test scores, from a pre-training average score of 6.8 (out of 10) to a post-training average score of 9.5.

Change in self-reported work practices: Between the first and second surveys, the research team found statistically significant and meaningful improvements for 10 of the 12 practices. These improvements were sustained when the third survey was conducted seven weeks after the training.

Change in claim rates: Between the 2012-2014 period and 2017, lost-time claim rates of the types of falls targeted by the training declined by 19.6 per cent—from 1.35 to 1.09 per 1,000 FTEs. Although the incidence rates of the other two claim categories also declined during the same period, these reductions were much smaller: 2.1 per cent for untargeted falls and 7.2 per cent for other acute injuries.

Reductions were largest among very small employers—those with fewer than five FTEs. Claim rates due to targeted falls among this group declined by 36.7 per cent over the study period, compared to a decline of 4.2 per cent for untargeted falls and 21.9 per cent for other acute injuries. In contrast, among firms with five to 49 employees and for firms with 50 or more employees, reductions in the claim incidence rates for targeted falls were 6.8 per cent and 12.3 per cent respectively.

This study provides evidence that a mandatory requirement for workers to complete a specified working-at-heights training was effective in reducing the incidence of fall injuries targeted by the training. From 2012-14 to 2017, the lost-time claim incidence rate for targeted falls declined by 19.6 per cent. Taking the decline in incidence rate for acute traumatic injuries not due to falls—7.2 per cent in the same period—as a measure of the larger trend in the sector, the difference—12 per cent in targeted falls—could be attributed to the regulatory intervention.

The finding that the intervention’s effect was greatest on the smallest firms may be an important finding. Small and medium-sized employers often lack the expertise or organizational policies and procedures to address OHS hazards. A mandatory prevention initiative may have an impact on safety practices at such organizations whereas voluntary initiatives may not.

A strength of this study is its ability to measure work-related injury outcomes in a complete population of workers under the jurisdiction of a single regulatory authority. Another strength is its inclusion of injury outcomes not targeted by the regulatory intervention for comparison. A weakness of the study is its inability to account for larger trends—such as changes in the business cycle or workforce composition—that may differently affect the three types of injury outcomes. Another potential weakness is its reliance on claims data, which may be vulnerable to under reporting of injury incidence.