If you were a busy practitioner seeking information on managing back pain, where would you turn: a blog by a person describing her experiences, a fact sheet from a reputable hospital, a research study in a scientific journal or a tabloid newspaper article?
We all apply a level of trust to information based on the source and the quality we associate with it. Plus, time demands and our access to information or our ability to understand it can also influence what we choose.
Scientists generally place the most trust in information published in journals that use the peer-review process. "Peer review" means that each study submitted to a journal is sent by its editors to two or three other experts in that field. These experts, or peers, provide an anonymous critique with a view to improve the write-up of the study. If they don't think the study meets certain scientific standards, they might advise against publishing it at all. Peer review helps to maintain scientific standards.
Practitioners in workplaces may not have access to peer-reviewed journals, or the time or expertise to wade through scientific text. They're more likely to turn to other sources of information that they trust. Examples could be trade publications, government reports, survey results from a polling company or technical reports.
These documents are all considered “grey literature.” The term grey literature comes from the uncertainty of the status of this information. Although there are several formal definitions, grey literature is essentially any document that hasn't gone through peer review for a publication. It can also include conference proceedings or doctoral theses.
Challenges with reviewing grey literature
When IWH reviewers conduct systematic reviews on a topic, they search for studies on that topic in peer-reviewed journals. We've found that practitioners who are consulted during reviews sometimes ask us to include the grey literature as well, to make sure that the search is as comprehensive as possible.
One concern of reviewers is the scientific quality of the studies. If an article doesn't go through peer review, it's possible for the author to make claims or interpretations that aren't supported.
Until recently, it has been more difficult to systematically search the grey literature than peer-reviewed studies. These documents often aren't indexed (or catalogued) in the major databases that are typically and systematically searched in reviews. It usually requires extra effort to find and get copies of these documents.
The format of a grey literature document can be quite diverse, unlike scientific papers that follow the structure of presenting background information, study methods, results and a discussion. So it's more difficult for reviewers to systematically extract information from grey literature as they do for peer-reviewed papers.
The benefits of reviewing grey literature
Grey literature documents can provide a richer source of detail than a scientific study. Because they aren't tied to a conventional structure, they can be longer and provide more detail. Research results can be written in a style that is more accessible and useful to a practitioner than a scientific paper.
Grey literature can also be published more quickly since it does not have to go through the potentially lengthy peer-review process. And in cases where there isn't much information on a topic in peer-reviewed research, grey literature may provide a valuable source of information.
Finally, grey literature is becoming easier to find. Increasingly, it is available on the Internet, and search engines and databases are providing ways of locating it.
Comparing results
In a recent review on participatory ergonomics, IWH researchers included grey literature documents in their review in response to requests from external practitioners. The end result? The findings from grey literature documents were similar to the peer-reviewed.
Grey literature can provide a systematic review with an additional source of rich information, depending on the topic and the nature of the research. The challenges and benefits need to be weighed against each other when deciding on whether to include it in a systematic review.
Source: At Work, Issue 52, Spring 2008: Institute for Work & Health, Toronto