Suppose you are a government official deciding what research studies should be funded. Or maybe you’re the editor of an academic journal, considering whether a study should be published. How would you know whether the science was valid?
Assessing the validity of a scientific study often requires specialized technical expertise on a range of methods, from sampling to data collection to analysis. And because scientific progress depends on new ideas and innovation, establishing appropriate standards and criteria to evaluate novel approaches can be difficult.
So research disciplines have turned to peer review: having other experts in the field judge the work of their professional equals or peers. The peer review process provides quality control throughout the life cycle of a research project. It is used by research funding agencies to decide which proposals should be given money to proceed. Many scientific conferences use peer review to select presenters. And academic journals use the process to evaluate papers submitted for publication.
Let’s take a closer look at the example of peer review at a scientific journal. Typically, when a paper is submitted, the journal editor will scan it first to make sure it complies with the journal’s guidelines on relevance, length and style. Then, this editor will look through the journal’s roster of willing reviewers to find researchers who have good knowledge of the subject matter. Most journals use at least two reviewers to assess each article submission.
Usually, the identity of the reviewer is not revealed to the author, although some open access journals publish reviewer names. Some journals will take an additional step and mask the identity of the study authors as well (a practice called double-blinding). This is to minimize the risk that a personal or professional relationship may influence the review. Where the identity of the author is not concealed, reviewers will declare any potential conflict of interest they may have. Many researchers will decline to review a paper if they’ve previously collaborated with the study authors or if they are colleagues at the same institution.
The job of reviewers is primarily to comment on the quality of the science. They consider whether the study design is appropriate for the research question, whether the methods used to recruit participants minimize the potential for bias and can be replicated, and whether the conclusions drawn by the study author are supported by the data. They also consider whether the work is novel or innovative, or makes an important contribution to scientific knowledge.
Based on these and other considerations, reviewers recommend rejecting the paper or accepting it—either as submitted or, most often, in a revised form that takes into account reviewer comments and suggestions. Sometimes reviewers will disagree with each other in their assessments of the study. The debate between reviewers can be vigorous, and is an important element for improving the quality of the science.
The peer review process has its challenges. Reviewers are busy researchers who aren’t paid for their reviews. With the volume of research and published studies on the rise, it can be difficult to find reviewers. However, many reviewers find they learn a great deal from participating in the peer review process. Many also like being up to date on new research and feeling part of the academic community.
Peer review is an important process by which scientists help each other improve their work. It also binds researchers into a community where they mutually rely on each other for thoughtful and constructive expert evaluation and feedback.
From the very inception of a study to the sharing of its findings, peer review helps ensure that a study meets quality standards before it becomes part of the scientific record.
Source: At Work, Issue 91, Winter 2018: Institute for Work & Health, Toronto