Skip to main content

Systematic Reviews: 10 Step roadmap

About this page

This page is inspired by the book, Boland, A., Cherry, M.G. and Dickson, R. (2017) Doing a systematic review: a student's guide. 2nd edn. London: SAGE. We have presented the systematic review workflow as a 10 step process, or roadmap. Remember! The process may not always be this linear and you may need to go back to start some of the stages again.

This is a training page based on the face to face course that we would normally deliver. This course is 180 minutes normally so lots of content to organise! Please look at the Pre-Introduction video below. (7.52 mins). All videos have captions by watching in full screen. 

Online Q & A sessions to complement this learning will be available on the 11th Sept, 9th October, 4th November, 14th December and 15th January 2021; Please book by Gradbook or email eprints@soton.ac.uk. You can test your learning by completing these 10 short exercises.

On this page

Step 1: Planning

Step 2: Scoping searches, review question and protocol

Step 3: Literature searching

Step 4: Screening

Step 5: Obtaining papers

Step 6: Choosing the full text

Step 7: Data extraction

Step 8: Quality assessment

Step 9: Synthesis

Step 10:Writing up

Test your learning 

 Post your questions on our padlet. We would like a different perspective in order to improve so please do add a comment/question. 

 

*** Stop Press*** Boland and Cherry 2nd edition is now available online. 

Step 1: Planning

Carrying out a systematic review will take longer than you might anticipate so planning is important. Consider the time and resources available to you. Consider using a Gantt chart as this will help you plan and monitor your progress. Take a look at this brief LinkedIn Learning course called 'Learning Gantt charts'.  Please note: You do not need to connect your LinkedIn account. You will need your university username and password.

There is a also an online course from John Hopkins University called an Introduction to Systematic Reviews and Meta Analysis. It is very comprehensive and takes about 13 hours to complete with a mixture of videos and exercise to do. It is free.

Include time to investigate the different databases and platforms that may cover your topic as you are likely to use more than one in your review.

Recorded presentations below, we suggest you watch them in this order. 

  • 1.1    Reporting Guidelines - Equator network/PRISMA/PRESS (5.40)
  • 1.2    Reference Management Software - Learn how to manage all the references you are going to find. (10.24) 
  • 1.3    Theory/Introduction to Systematic Reviews (7.21)
  • 1.4    Bibliographic Databases. What are they, which ones to use and how to find them (8.58)

Step 2: Scoping searches, review question and protocol

As soon as you have an idea for your systematic review topic, you should run a scoping search. This will enable you to see what has been published already, if anything. This stage will take you longer than you imagine, so give yourself plenty of time. Please watch video below on Starting/Scoping first for some context.

Questions to ask yourself:

  • Has a systematic review already been published on this? If yes, when? If it was more that 5 years ago then you might find enough new evidence to update the review.
  • Is there enough evidence to conduct a systematic review? If you can only locate 2 research papers on your topic then a meaningful systematic review is unlikely.
  • I am finding too much primary research? If yes, consider how you can focus your question.

Recorded presentations below

  • 2.1   Scoping/Starting your search - what resources to use (3.15)
  • 2.2   Scoping your search using the Trip database (3.24)
  • 2.3   Delphis  - when and how to use it (6.33)
  • 2.4   Framing your question (4.59)
  • 2.5   Protocols/Proposals/PROSPERO (5.26)

Review Question and Protocol

As part of the process you will need to finalise your review question (see Frameworks) and agree the exclusion and inclusion criteria for your review. This forms part of your protocol

Step 3: Literature searching

Having completed your scoping searches you should now be aware of all the terms that you want to include and these should be finalised. We would recommend refining your search strategy (the terms and how they will be combined) on one database and platform, checking that all members of the team agree that it is complete and thorough, before running it on another database. Moving to a different database may involve translating the search strategy and syntax, for example, identifying the thesaurus term used or changing the symbol for truncation (* to $).

The recorded presentations below include

  • 3.1   Overview of Thesaurus searching/Controlled Vocabulary/Subject Headings/MESH/Descriptors (they all describe the same idea)
  • 3.2   OVID platform/Medline database using keywords and Thesaursus/MESH headings -  "belt and braces" approach
  • 3.3   EBSCO platform/PsycINFO database using keywords and Descriptor searching  - "belt and braces" approach (15.14)
  • 3.4   Grey literature  - Equally important.(3.29)
  • 3.5   Help adapting search strategy for use on database platforms, eg OVID and EBSCO (3.15)
  • 3.6   Testing your search strategy. (3.32)

Before you start your final search process and importing into your reference managment software. Please Test, Test and Test again. Short video (3.32 min) on how to do this and why so important. 

Step 4: Screening

When you have completed your literature searches, you will want to remove any duplicate records and start to screen the results. You will be using your inclusion and exclusion criteria to select or exclude articles. You will need to keep a record of this process to enable you to produce the PRISMA flow diagram.

Review the records first, scanning the title of the article, the abstract and any subject descriptors, constantly referring back to your exclusion/inclusion criteria.

You may find it helpful to use the reference management software package called EndNote to manage this process. Our EndNote for systematic reviews page will show you how to export large numbers of results.

You can organise your search results into groups by database name, EndNote will help remove the duplicates, you can then export the remaining results for screening into Excel, or the free online screening program called Rayyan. Please see this comprehensive guide on how to use it by McGill University 

Step 5: Obtaining papers

Check eJournal Title search for journal availability in our online collection, you will need to be connected to the University using the Virtual Private Network (VPN) called GlobalProtect.

If the journal title is not available please submit an Inter-Library Loan (ILL) request via WebCat; the process is...

  1. Login to WebCat http://www-lib.soton.ac.uk
  2. Select ‘Requests & Document Supply’
  3. Followed by ‘Request something not available in our collections’
  4. Then, ‘Requests for Journals/Articles (including pages from)’
  5. Complete the online form
  6. In the box labelled School/NHS/NERC please add ‘your department name’ e.g. medicine (you do not need a fund code)
  7. Then select ‘Place Request’

Further information about the ILL service can be found here

This video  (7 .35) takes you through the process of finding full text. If you are using a reference manager you may be able to use it to find full text as long as you are 'on campus' (GlobalProtect VPN)

Step 6: Choosing full-text

Once you complete the screening of the records and have the full-text of the papers you want to review, apply your inclusion and exclusion criteria again as your read the full article. An approach you may want to consider is to read the abstract, the introduction and conclusion and only then read the whole paper, if it still meets the inclusion criteria.

Step 7: Data extraction

Data extraction can start when you have selected your articles for review. You will need to extract and examine the data. For a full explanation, look at chapter 5, Higgins JPT, Thomas J, Chandler J. Collecting data. In: Li T, Higgins JPT, Deeks JJ, et al. (eds.) Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. Hoboken, NJ: Wiley Blackwell; 2019 p109-41. https://training.cochrane.org/handbook/current/chapter-05.

Take a look at the video on the companion site to Boland A, Cherry MG, Dickson R. Doing a systematic review: a student's guide. 2nd ed. London: SAGE; 2017. called, "Chapter 6: Data Extraction: Where Do I Begin?".

Another chapter 6! But this time in Bettany-Saltikov J. How to do a systematic literature review in nursing a step-by-step guide. Maidenhead: Open University Press; 2012. http://lib.myilibrary.com?id=367986 gives a clear introduction to the process, and includes an example data extraction template that you can adapt.

Step 8: Quality assessment

Critical appraisal tools

The Centre for Evidence Based Medicine (CEBM) provides some useful guidance and a checklist for systematic reviews

See also our section on critical appraisal  - at the bottom of the page.

There are some useful resources from the University of Edinburgh as well as Cardiff University  

Step 9: Synthesis

More useful resources from the University of Edinburgh. Please see powerpoint on Data extraction, Quality Assessment and Writing up.

Helpful guides from the Institute of Education as well as the University of Queensland 

 

 

Step 10: Writing up

If you have not already, please look at the video - 1.1 Guidelines for reporting Systematic Reviews which explain the Equator Network for reporting guidelines and PRISMA ( both essential resources)

What is a reporting Guideline

Guidance on scientific writing 

Reporting guidelines for Systematic Reviews

Please see powerpoint on "writing up" by Prof Wardlaw University of Edinburgh. (link to more general advice on writing papers) 

Test your learning

Please feedback on the Padlet if you want to clarify any issues or understanding.