Skip to main content
Jefferson

Systematic Reviews

Resources for planning and conducting a Systematic Review of the literature.

Definition

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question.  It  uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993). The key characteristics of a systematic review are:

  • a clearly stated set of objectives with pre-defined eligibility criteria for studies;
  • an explicit, reproducible methodology;
  • a systematic search that attempts to identify all studies that would meet the eligibility criteria;
  • an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and
  • a systematic presentation, and synthesis, of the characteristics and findings of the included studies. (Source: Cochrane)

Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies. Meta-analysis refers to the use of statistical techniques in a systematic review to integrate the results of included studies. (Source: PRISMA)

Avoid Unnessary Duplication

Before embarking on a systematic review, perform a literature search to assess whether published systematic reviews and protocols have been written on the same topic. A few examples of databases to search for this purpose are provided below. 

PROSPERO: an international database of registered systematic reviews and protocols in health and social care

  • Title search: use the simple search for words or phrases 
  • Filtered search: in addition to searching with MeSH terms, selecting the “Show filters” button provides six different categories to choose from
  • Cochrane protocols: “Key details from new Cochrane protocols are automatically uploaded into PROSPERO”  

Cochrane Library:

  • Search results may be filtered for (Systematic) Review or Protocol by selecting the relevant radio button – All (43 results) is the search result default:

PubMed:

Systematic review search: utilizing the Article types filter will limit to systematic reviews.​ Note: other types of reviews (e.g., literature review) may also be retrieved.

Activating the filter:

  • Perform a search – the example below is for fibromyalgia
  • Select Customize to display the Article types box
  • Tick the box next to Systematic Reviews and select Show

  • Check the Systematic Reviews link to activate and PubMed will automatically display the updated results:

Protocol search: same filtered search as above plus AND protocol:

Note: this search technique is inexact and will possibly retrieve systematic reviews including studies employing other types of protocols, as indicated by viewing the abstract for result #1 above (not shown here). 

Journal Selection

Once it has been decided that a systematic review will be conducted, this may be the time to identify potential journals which may be the most relevant. Journal editors will judge a systematic review based upon various parameters, including how it may add value to the scientific knowledge base, change clinical practice, fit within the journals’ mission and whether it complies with a particular standard such as PRISMA or from the National Academy of Medicine. Familiarity with the publication’s “instructions to authors” may save time in this regard. The Journal of the American Medical Association provides an excellent example of formatting and recommendations for systematic reviews.

Writing a Protocol

Producing a protocol as the “plan” for the systematic review is recommended, as it enhances communication between team members, reduces potential for bias, and provides the research community with an informed view of the eventual systematic review process, if published. Guidance for formatting a protocol can be found below.

PRISMA-P:  Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols

PROSPERO:

Cochrane Protocols:

Protocol example incorporating the PRISMA-P format and registration in PROSPERO:

  • Downes MJ, Mervin MC, Byrnes JM, Scuffham PA. Telemedicine for general practice: a systematic review protocol. Systematic Reviews. 2015 Oct 5; 4:134.

Why register a systematic review:

           

Updating a Systematic Review

A team discussion on whether the systematic review will be updated in the future is helpful. Currently, there seems to be no consensus on when and how this process should be accomplished. Some brief points to consider are:

  • Who will lead the update in the future and how will its necessity be determined?
  • Which bibliographic management system will the references from the first systematic review be held?
  • How will the search strategy be updated?

Select references:

  1. K. G. Shojania, M. Sampson, M.T. Ansari, J. Ji, C. Garritty, T. Rader, and D. Moher. Updating Systematic Reviews:Technical Reviews, No. 16. http://www.ncbi.nlm.nih.gov/books/NBK44099/
  2. Shekelle PG, Motala A, Johnsen B, Newberry SJ. Assessment of a method to detect signals for updating systematic reviews. Systematic Reviews. 2014 Feb 14;3:13. ​http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3937021/
  3. Garritty C, Tsertsvadze A, Tricco AC, Sampson M, Moher D. Updating systematic reviews: an international survey. PLoS One. 2010 Apr 1;5(4):e9914. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2848577/