An eight-item questionnaire developed by the Institute for Work & Health (IWH) has now been adopted by WorkSafeNB as a benchmark tool to measure occupational health and safety culture among employers in New Brunswick.
WorkSafeNB is responsible for both workers’ compensation and health and safety enforcement in New Brunswick. It recently teamed up with IWH to validate two safety culture assessment tools, including a questionnaire the IWH had developed in 2009.
That questionnaire, called the Organizational Performance Metric (OPM), was developed by IWH and Ontario’s prevention system partners to measure leading indicators of health and safety performance in workplaces.
Validation
In WorkSafeNB’s joint validation study with IWH, the OPM was shown to correlate with historical claims rates. In other words, the OPM could correlate a firm’s past workplace injury experience with the firm’s perception of its current health and safety policy and practices. The finding suggests the OPM may also be able to predict future performance. A summary of the study, recently published by IWH, can be found here.
That exercise validated the OPM,
says Barb Keir, director of Program Development and Evaluation at WorkSafeNB. And that’s why we moved to adopt it.
The decision meets a long-standing need at WorkSafeNB for a short and easy tool to assess key elements of safety culture in an organization. Since 2001, WorkSafeNB had been using a lengthy health and safety infrastructure measurement tool to do the job. That perception tool had questions covering five fundamentals and 22 topics. It had to be administered to all employees, supervisors and managers in an organization, making the process very time-consuming.
That survey was too complex and cumbersome,
says Anne Lise Albert, assistant director of Program Development and Evaluation at WorkSafeNB.
Two tools studied
The team did a scan across the country and found six surveys similar to what it had in mind, including the Institute’s OPM. WorkSafeNB decided to create its own tool based on the six, and the result was an 18-item questionnaire called the Internal Responsibility System Questionnaire (IRSQ).
Given that the Institute had the expertise and experience validating its OPM, WorkSafeNB asked a research team at the IWH, headed by Senior Scientist Dr. Benjamin Amick, to conduct a validation study of its IRSQ tool. The research team also took the opportunity to further study IWH’s previously validated OPM.
Both tools were sent out to about 800 employers. These were asked to have an employee, plus either a senior manager or a supervisor, fill out both questionnaires.
About 250 firms responded to the full request, and another 80 had only one representative complete the questionnaires. WorkSafeNB then made its de-identified historical claims dataset available to IWH researchers to assess how the results of each tool were associated with past injury claims.
The results showed a strong correlation between OPM scores and five-year historical claims rates. Firms that had better OPM scores also had lower claims rates, including both lost-time and no-lost-time claims. The analyses suggest that firms that move up the OPM scorecard from weak to strong (from the lowest scoring to the highest scoring of the OPM’s four categories) could reduce total claims by over 30 per cent.
In contrast, the results found a weak correlation between the IRSQ and five-year workers’ compensation claims rates. Michael Swift, a member of the IWH project team, said more research would be needed to determine why the OPM tracked well with compensation claims but the IRSQ did not. The two tools cover generally the same areas, with one key difference being the answer scales. In the IRSQ, respondents were asked about the extent to which they agreed or disagreed with a description of a health and safety practice. In the OPM, however, respondents were asked what percentage of time the practice took place at their firm.
That’s a key difference between the two questionnaires,
says Swift. And while we would need more research to know why the IRSQ didn’t work, the answer may lie in that difference.
Further questions
But Keir says she and her team at WorkSafeNB are not focused on why the IRSQ was not validated. Our goal was to have an efficient and accurate tool.
Given that the OPM has been twice validated, she adds, WorkSafeNB will use it as a benchmarking tool for the province’s employers.
I’m very happy that we now have this easy and short tool to use in our New Brunswick workplaces,
says Albert. I’m also extremely pleased with this relationship that we’ve got with the Institute. It has been a fantastic partnership.
On his part, Amick says his team still needs to do further research on the tool. What do respondents think about when they choose their answers? And what does it mean inside the workplace to move an organization from the bottom of the scale to a higher level?
We are excited New Brunswick is an early adopter of the OPM. We have a great opportunity to provide people with a short and easy-to-use tool,
says Amick. This is just the beginning of the development and validation of a new tool.