Occupational health and safety management systems (OHSMSs) have developed considerably over the last 20 years. There are now more OHSMSs available and in place than ever before. Yet little is known about the effectiveness of these systems on employee health and safety and on relevant economic outcomes.
A systematic review of the literature was undertaken to see what existing research has found about the effectiveness of OHSMSs. Many countries, including Canada, are in the process of developing management standards for occupational health and safety, so a better understanding of the impact of these systems is timely.
The review team set out to investigate three key questions:
- What is the relative effectiveness of mandatory and voluntary OHSMSs on employee health and safety and on associated economic outcomes?
- What facilitators and barriers are there to the adoption and the effectiveness of OHSMSs?
- What is the evidence on the cost-effectiveness of OHSMSs?
What are OHSMSs?
There is currently no common understanding in the literature on what is meant by an “OHSMS” and how to distinguish it from other occupational health and safety (OHS) programs. The review team developed the following definition:
“An OHSMS is the integrated set of organizational elements involved in the continuous cycle of planning, implementation, evaluation, and continual improvement, directed toward the abatement of occupational hazards in the workplace. Such elements include, but are not limited to, organizations’ OHS-relevant policies, goals and objectives, decision-making structures and practices, technical resources, accountability structures and practices, communication practices, hazard identification practices, training practices, hazard controls, quality assurance practices, evaluation practices, and organizational learning practices.”
OHSMSs are generally more proactive than other types of OHS programs. They are also better integrated internally, and they incorporate more elements of evaluation and continuous improvement.
The systematic review looked at evidence involving both voluntary and mandatory OHSMSs.
Voluntary OHSMSs usually arise through private enterprise, employer groups, government and its agencies, insurance carriers, professional organizations, or standards associations and are not directly linked to regulatory requirements. Government-affiliated agencies or insurance carriers sometimes offer incentives to organizations that adopt particular voluntary OHSMSs. An example of this would be the U.S. chemical industry’s Responsible Care system, a performance initiative aimed at a number of outcomes, including reduction in workplace injuries.
Mandatory OHSMSs are usually implemented because of government legislation and are enforced through inspections, fines, etc. In general, mandatory OHSMSs have simpler requirements since they are intended for use in most workplaces, including small workplaces. An example of this would be the Internal Control Legislation from Norway. This legislation requires all workplaces take a systematic approach to health and safety.
What is a systematic review?
In doing a systematic review, researchers develop a clearly formulated question, use systematic and explicit methods to identify, select and critically appraise relevant research, and then analyze data from studies selected in the review process. The review normally includes the following steps:
- determining the question
- developing a search strategy and searching the literature
- selecting studies that meet inclusion/exclusion criteria
- assessing the methodological quality of selected studies and eliminating those in which quality is not sufficient
- systematically extracting and summarizing key elements of the included studies
- describing the results from individual studies
- synthesizing the results and reporting them.
What research was included in this systematic review?
The review team searched seven electronic databases covering a wide range of journals. These contained mainly abstracts of peer-reviewed articles from a variety of disciplines.
How did the review proceed?
Literature search
Reviewers sought relevant studies on OHSMSs, including both implementation and effectiveness research. The initial search produced 4807 studies.
Study relevance
The studies were screened for relevance by testing their titles and abstracts against a set of inclusion criteria. One of the criteria was that the intervention involve at least two elements of OHSMSs, of which one had to be a management element. The team then reviewed the full articles of those that passed the initial screening, again using the inclusion criteria. At this point, 18 studies were considered eligible and were appraised for their methodological quality.
Quality appraisal, data extraction and evidence synthesis
The methodological quality of each study was rated independently by at least two reviewers using a set of explicit criteria. The reviewers then met to reach consensus.
For evidence to be included in the data extraction and evidence synthesis phases of the review, reviewers had to agree it met the standard for being of at least “moderate” quality. Nine studies achieved this minimal quality requirement.
Of the nine studies, four examined voluntary systems and five evaluated mandatory systems. None of these studies provided evidence of sufficient quality on facilitators and barriers, but they did provide information on the implementation and effectiveness of OHSMSs.
Findings
Voluntary OHSMSs
All studies involving voluntary OHSMS interventions reported positive findings. While the outcomes measured varied among studies, the findings included increased implementation over time, better safety climates, increased hazard reporting by employees, more organizational action taken on occupational and health issues and decreased workers’ compensation premiums. It is likely that the size of the observed declines in premium rates (23 and 52 percent) would be considered important by stakeholders.
However, all the studies in this group were of only “moderate” quality, largely because their study designs were so simple. Because of this, the reviewers had some doubts about the validity of the findings. Three of the four studies involved single workplaces, which makes the applicability of the findings to other workplace settings uncertain. There were also reasons to suspect that publication bias might account for the consistently positive findings in these studies.
Mandatory OHSMSs
All five studies involving mandatory OHSMSs reported positive findings. Some documented increased OHSMS implementation over time. Others found that OHSMS implementation improved how workers perceived both the physical and psychosocial working environment. It also increased workers’ participation in health and safety activities, reduced rates of lost-time injury and led to increases in productivity. It is likely that the size of the observed changes in OHSMS implementation and injury rate would be considered important by stakeholders.
All the studies in this group had moderate methodological limitations, mainly linked to the simplicity of the study designs.
No studies were found that compared voluntary and mandatory OHSMS interventions directly. No good quality studies that looked at facilitators of and barriers to OHSMS implementation or effectiveness were found. The reviewers were also unable to find studies which estimated the cost of implementing OHSMSs in workplaces.
Strengths and limitations of this systematic review
Strengths of the review
The team collaborated with key stakeholders to formulate questions for the review, which should increase the relevance of the findings among these groups. The team then reviewed studies drawn from a broad range of disciplines. The search confirmed that no one else has carried out a systematic review of the evidence looking at the effectiveness of OHSMSs.
Limitations of the review
There was a large volume of literature in the seven databases. This meant reviewers could only carry out a preliminary search of literature that has not been peer-reviewed and published. A search of thesis dissertations and “grey literature” did come up with a number of government reports and publications which met the inclusion criteria. More research is needed to determine whether the quality of such material is good enough to contribute to the evidence base on OHSMS.
Identifying and addressing research gaps
The review identified a number of gaps in the research. The most important was the lack of research whose explicit purpose was to study the effectiveness of voluntary and mandatory OHSMSs on employee health and safety and economic outcomes. The research designs used in the selected studies were not rigorous enough to produce great confidence in the findings. This lack of high-quality studies may reflect, at least in part, how difficult it is to carry out applied research in workplaces.
Conclusions and recommendations
This synthesis of the best evidence available showed consistently positive effects in workplaces for voluntary and mandatory OHSMSs. However the absolute number of studies producing these results was not large, and the quality of the studies was not high. The team also questioned the applicability of these results to Canadian workplaces.
There is insufficient evidence in the published, peer-reviewed literature on the effectiveness of OHSMSs to make recommendations either in favour of or against OHSMSs. This is not to say such systems are ineffective or undesirable, but rather to suggest we should be cautious based on existing research knowledge.
Given the current state of evidence regarding OHSMSs effectiveness:
The review team recommends that those who fund Canadian research should support studies examining the effectiveness or cost-effectiveness of OHSMSs.
Support should also be given to research aimed at identifying facilitators of and barriers to OHSMS implementation and effectiveness.
The usefulness of this research would be greatly enhanced if stronger research designs were employed. This would include the use of comparison groups and longitudinal designs. It is also important for researchers to carefully identify and control for potential confounders, to use larger samples selected through random means, and to include more rigorous economic evaluations. Research using both qualitative and quantitative methodologies would be helpful.
The review team recommends that when the topic of OHSMSs is reviewed in future, researchers seek evidence from sources outside the peer-reviewed, published literature.
This review involved an extensive search for studies on occupational health and safety management systems. It focused on the published, peer-reviewed literature in order to concentrate on high-quality studies. However, relatively few studies were found and they were only of “moderate” quality. Others who are interested in doing similar research should consider using additional sources of literature.
Because all mandatory and some voluntary OHSMSs are initiated by the government and its agencies, these organizations are a likely source for evaluative reports on the subject. Thesis dissertations might also provide a valuable pool of high-quality studies, since their production involves peer-review.