Systematic review

About the “What researchers mean by...” series

This research term explanation first appeared in a regular column called “What researchers mean by…” that ran in the Institute for Work & Health’s newsletter At Work for over 10 years (2005-2017). The column covered over 35 common research terms used in the health and social sciences. The complete collection of defined terms is available online or in a guide that can be downloaded from the website.

Published: August 2014

Think of the last time you came across a research study that seemed to contradict some other study on the same question. You can probably think of a few examples, especially for health topics that are often in the news. One moment you hear that acupuncture helps relieve pain. The next, a new study says it doesn’t.

If you think about how research studies are conducted, you can appreciate why discrepancies in findings arise. Different researchers studying the same question might enlist different numbers of participants. They might choose different study designs. There might be differences in how they administer the treatment or intervention or how they measure the effect of the intervention. All these things make a difference to what researchers ultimately find.

In other words, when looking for research evidence, you need to look beyond a single study and take into account the overall body of evidence. But given the amount of published research on a given topic, keeping up on the evidence can overwhelm anyone—including clinicians, researchers and policy-makers.

This is where systematic reviews come in. They help people keep up on what the overall body of research says on a topic. They’re designed to take into account the reliable available evidence on a subject at a given point in time.

To do this, researchers on a systematic review team go through all the studies relevant to a topic and assess the quality of each. From the higher quality studies, they’ll pull out a synthesis of the findings. Often, they’ll combine the data from different studies to do what’s known as meta-analysis. And as systematic reviews can only synthesize the available research at a point in time, they need to be updated regularly.

Narrative versus systematic reviews

To better understand systematic reviews, consider traditional narrative reviews that were once more commonplace. Like systematic reviews, narrative reviews also synthesize the scientific literature on a given question. The main difference is narrative reviewers draw chiefly from their experience and expertise for their analysis. This makes narrative reviews more susceptible to bias. No clear methodology is evident to help readers understand whether reviewers have considered all the available evidence, or how and why they recommend one study over another.

Systematic reviews, in contrast, minimize this type of bias by putting methodology front and centre. Like any other scientific study, systematic reviews should be replicable. That means another research team, using the same methodology to tackle the same question, should be able to gather the same evidence and come to the same conclusion.

As such, all the steps taken in systematic reviews are clearly and transparently outlined. Right from the literature search, systematic reviews spell out what terms are used, which databases are searched, and what criteria are applied to limit the search (e.g. language of published studies). Subsequent steps are guided just as much by methodology—from deciding what studies are relevant, to assessing the studies for how rigorously they were carried out.

Another distinguishing aspect of systematic reviews is their focus. While narrative reviews might cover off a broad topic, systematic reviews centre on a single research question. This question is typically defined by applying the PICO principle; that is, the question indicates the population, intervention, comparison and outcome being considered in the review. The result might read like this statement of objective from an actual review: “A review of randomized trials of acupuncture for adults with non-specific (sub)acute or chronic low-back pain.”

Systematic reviews, though relatively new, are growing more popular as people increasingly recognize the value of evidence-based practice and policy. Given the amount of new research being produced, systematic reviews have become an important tool for staying up to date.

Source: At Work, Issue 77, Summer 2014: Institute for Work & Health, Toronto