Systematic Reviews: 10 Step roadmap

Carrying out a systematic review will take longer than you might anticipate so planning is important. Consider the time and resources available to you. Consider using a Gantt chart as this will help you plan and monitor your progress. Take a look at this brief LinkedIn Learning course called 'Learning Gantt charts'.  Please note: You do not need to connect your LinkedIn account. You will need your university username and password.

There is a also an online course from John Hopkins University called an Introduction to Systematic Reviews and Meta Analysis. It is very comprehensive and takes about 13 hours to complete with a mixture of videos and exercises to do. It is free.

Include time to investigate the different databases and platforms that may cover your topic as you are likely to use more than one in your review. Also decide on what reference management software you are going to use. We support Endnote but there are other options. 

Recorded presentations below, we suggest you watch them in this order. 

• 1.1    Reporting Guidelines - Equator network/PRISMA/PRESS (5.40)
• 1.2    Reference Management Software - Learn how to manage all the references you are going to find. (10.24) 
• 1.3    Theory/Introduction to Systematic Reviews (7.21)
• 1.4    Bibliographic Databases. What are they, which ones to use and how to find them (8.58)

As soon as you have an idea for your systematic review topic, you should run a scoping search. This will enable you to see what has been published already, if anything. This stage will take you longer than you imagine, so give yourself plenty of time. Please watch video below on Starting/Scoping first for some context.

If you are new to searching health and medicine databases or would like a refresher, please listen to a Basic and Advanced recording of a search in CINAHL. The advanced recording covers using MESH or subject headings. 

Basic recording  27 Mins

Advanced recording 10 Mins 

Questions to ask yourself:

• Has a systematic review already been published on this? If yes, when? If it was more that 5 years ago then you might find enough new evidence to update the review.
• Is there enough evidence to conduct a systematic review? If you can only locate 2 research papers on your topic then a meaningful systematic review is unlikely. Consider looking at Connected Papers  - a simple network analysis based on similarities between papers. 
• I am finding too much primary research? If yes, consider how you can focus your question.

Recorded presentations below

• 2.1   Scoping/Starting your search - what resources to use (3.15)
• 2.2   Scoping your search using the Trip database (3.24)
• 2.3   Library Search video - to come (1/09/22)
• 2.4   Framing your question (4.59)
• 2.5   Protocols/Proposals/PROSPERO (5.26)

Review Question and Protocol

As part of the process you will need to finalise your review question (see Frameworks) and agree the exclusion and inclusion criteria for your review. This forms part of your protocol

Having completed your scoping searches you should now be aware of all the terms that you want to include and these should be finalised. We would recommend refining your search strategy (the terms and how they will be combined) on one database and platform, checking that all members of the team agree that it is complete and thorough, before running it on another database. Moving to a different database may involve translating the search strategy and syntax, for example, identifying the thesaurus term used or changing the symbol for truncation (* to $). Please see this useful guide from University of Alberta as well as another comprehensive guide from Cornell University on how to translate search strategies with links to useful tools. 

The recorded presentations below include

• 3.1   Overview of Thesaurus searching/Controlled Vocabulary/Subject Headings/MESH/Descriptors (they all describe the same idea) (17)
• 3.2   OVID platform/Medline database using keywords and Thesaursus/MESH headings -"belt and braces" approach  (3.04)
• 3.3   EBSCO platform/PsycINFO database using keywords and Descriptor searching  - "belt and braces" approach (15.14)
• 3.3a Field Codes  - using EBSCO/Medline (5.19) 
• 3.4   Grey literature  - Equally important.(3.29)
• 3.4a Proquest Dissertations and Thesis  - Grey literature (7.56)
• 3.5   Help adapting search strategy for use on database platforms, eg OVID and EBSCO (3.15)
• 3.6   Testing your search strategy. (3.32)

Before you start your final search process and importing into your reference managment software. Please Test, Test and Test again. Short video (3.32 min) on how to do this and why so important. 

When you have completed your literature searches, you will want to remove any duplicate records and start to screen the results. You will be using your inclusion and exclusion criteria to select or exclude articles. You will need to keep a record of this process to enable you to produce the PRISMA flow diagram.

Review the records first, scanning the title of the article, the abstract and any subject descriptors, constantly referring back to your exclusion/inclusion criteria.

You may find it helpful to use the reference management software package called EndNote to manage this process. Our EndNote for systematic reviews page will show you how to export large numbers of results. Please see video conversation below for more information on how to do this. There are more video conversations on our EndNote training pages. You can then export the remaining results for screening into Excel, or a subscription tool such as Rayyan

There are some free systematic review tools have a duplicate detection and screening functions such as The Systematic Review Accelerator (SRA) and HubMeta.

Check eJournal Title search for journal availability in our online collection, you will need to be connected to the University using the Virtual Private Network (VPN) called GlobalProtect.

If the journal title is not available please submit an Inter-Library Loan (ILL) request via Library Search; the process is...

1. Login into Library Search 
2. Select Requests from your My Account (top right hand side)
3. Select Article 
4. Complete the online form
5. Read and Tick Copyright Acknowledged box 
6. Select Submit request 

Further information about the ILL service can be found here

This video has been removed as we transition to a new Library Management System.Aug 2022. if you require any help finding full text articles please email libenqs@soton.ac.uk
 

Once you complete the screening of the records and have the full-text of the papers you want to review, apply your inclusion and exclusion criteria again as your read the full article. An approach you may want to consider is to read the abstract, the introduction and conclusion and only then read the whole paper, if it still meets the inclusion criteria.

Data extraction can start when you have selected your articles for review. You will need to extract and examine the data. For a full explanation, look at chapter 5, Higgins JPT, Thomas J, Chandler J. Collecting data. In: Li T, Higgins JPT, Deeks JJ, et al. (eds.) Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. Hoboken, NJ: Wiley Blackwell; 2019 p109-41. https://training.cochrane.org/handbook/current/chapter-05.

Take a look at the video on the companion site to Boland A, Cherry MG, Dickson R. Doing a systematic review: a student's guide. 2nd ed. London: SAGE; 2017. called, "Chapter 6: Data Extraction: Where Do I Begin?".

Another chapter 6! But this time in Bettany-Saltikov J. How to do a systematic literature review in nursing a step-by-step guide. Maidenhead: Open University Press; 2012. Working with your primary papers gives a clear introduction to the process, and includes an example data extraction template that you can adapt.

Critical appraisal tools

The Centre for Evidence Based Medicine (CEBM) provides some useful guidance and a checklist for systematic reviews

See also our section on critical appraisal on our sister page - scroll to the bottom of the page - in "Equally important" section.

There are some useful resources from the University of Edinburgh as well as Cardiff University 

Useful resource for reviewing studies with diverse designs  = QATSDD.   

Risk of Bias 

There are some guides to help you understand which tools to use

Overview of tool from Himmelfarb Health Sciences Library 

Article looking at tools used in Prospero.

Very useful overview from National Health and Medical Research Council Australia 

Please see useful website that give overview of the Cochrane tools.

• ROB 2 is the Cochrane risk-of-bias tool for randomized trials
• The Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool is recommended for assessing the risk of bias in non-randomized studies of interventions included in Cochrane Reviews.

More useful resources from the University of Edinburgh. Please see powerpoint on Data extraction, Quality Assessment and Writing up.

Helpful guides from the London School of Hygiene and Tropical Medicine.

If you have not already, please look at the video - 1.1 Guidelines for reporting Systematic Reviews which explains the Equator Network for reporting guidelines and PRISMA ( both essential resources)

What is a reporting Guideline

Guidance on scientific writing 

Reporting guidelines for Systematic Reviews

Please feedback on the Padlet if you want to clarify any issues or understanding.