About Systematic Reviews
Creating a Systematic Review Definition
To define a systematic review, we must first understand what it is not.
Systematic reviews can sometimes be confused with other kinds of reviews used to conduct evidence synthesis. For example, you should understand the differences between a systematic review vs meta-analysis, and a systematic review vs a literature review. You can check out our articles at the link to learn more.
To add to the confusion the results of a systematic review can be analyzed and synthesized quantitatively, or qualitatively. However, one should note that the differences in quantitative and qualitative systematic review methodology do not alter the basic structure of a systematic review. Let’s explore the definition of a systematic review in further detail.
Why Do We Need To Define a Systematic Review?
The term evidence-based medicine (EBM) was coined in the year 1990. Evidence-based medicine involves the analysis of the best available clinical evidence that is collected using a systematic search to guide teaching and practicing clinical medicine .
When it comes to looking for the best available evidence for treatment, randomized control trials (RCTs), systematic reviews, and meta-analyses are considered the gold standard .
Evidence synthesis is widely used in supporting clinical guidelines and making recommendations for practice. However, more recent evidence-based medicine is being looked at as a movement in crisis since there is just “too much available evidence” .
A study published in 2016 showed that over 8000 systematic reviews were being indexed annually in MEDLINE, corresponding to a three-fold increase over the last decade . This increased rate of systematic review publication indicates that some of them may be redundant, misleading, and conflicting. Therefore, it is evident that there is a lack of a precise definition leading to the inappropriate inclusion of studies as systematic reviews. A precise definition of a systematic review is required because without one, authors might call their review a systematic review when it actually does not meet the criteria for being termed so.
The Definition of a Systematic Review
The most commonly used sources for systematic review definitions are manuscripts describing the PRISMA statement and Cochrane Handbook [5, 6].
In section 1.2.2 of the Cochrane Handbook, titled What is a systematic review?, the following definition can be found, “A systematic review attempts to collate all empirical evidence that fits the pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993).
The key characteristics of a systematic review are: a clearly stated set of objectives with pre-defined eligibility criteria for the studies; an explicit, reproducible methodology; a systematic search that attempts to identify all the studies that would meet the eligibility criteria; an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias; and a systematic presentation, and synthesis of the characteristics and findings of the included studies”.
Although one could argue that this definition can be considered the formal definition of systematic reviews, there are terms that remain vague and ambiguous. They are open to interpretation by the authors. There could be studies where the methodology is explicit or reproducible, but not effective.
Reproducibility is the ability for the systematic review to be re-conducted, using the same methods and data by a different researcher or team. While there is no clear explanation and guidelines for defining “explicit” and “systematic,” researchers must make sure that they use evidence-based approaches to select methods with a view of minimizing bias and providing more reliable findings from which authoritative inferences can be made.
There is a need for the research community to develop a specific, and unambiguous definition of a systematic review. It can be formulated using the elements that make a systematic review as follows,
A systematic review is a review that includes the following:
- A clearly formulated research question.
- A pre-specified protocol that includes a reproducible search strategy used to collect primary resources (with all the details of the search strategy).
- Pre-defined, and strict inclusion and exclusion criteria for the studies included in the review.
- Selection methods – Dual independent screening of collected primary studies
- Critical appraisal of the included studies in the form of a risk of bias assessment
- Appropriate representation of data analysis, and synthesis which allows reproducibility of the results .
- Sur RL, Dahm P. History of evidence-based medicine. Indian journal of urology: IJU: journal of the Urological Society of India. 2011;27(4):487–9.
- Group EW. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420–5.
- Greenhalgh T, Howick J, Maskrey N. Evidence-based medicine: a movement in crisis? BMJ. 2014; 348:g3725.
- Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catala-Lopez F, Li L, Reid EK, Sarkis-Onofre R, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016; 13(5):e1002028.
- Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? Jama. 1998;280(3):280–2.
- Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7):e1000097.
- Krnic Martinic, M., Pieper, D., Glatt, A. et al. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol 19, 203 (2019).