Skip to Main Content

Systematic Reviews: Training - Health

Systematic Review training at University of Southampton

This page provides links to resources recommended in the Systematic Reviews training courses that we deliver at the University of Southampton.  They are not exhaustive and when you carry out your systematic review it is important that you identify all the relevant specialist databases and sources in your discipline. 

If you are new to searching health and medicine databases or would like a refresher, please listen to a Basic and Advanced recording of a search in CINAHL. The advanced recording covers using MESH or subject headings. 

Basic recording  10 Mins

Advanced recording 27 Mins 

Systematic Review - Health Training - Getting started.

Links to free online resources you can use to help you with the whole process of a Systematic Review.

Cochrane Handbook of Systematic Reviews. Freely available online Version 6 is publicly available from October 2019. (We have this as an ebook also so you can download sections as PDFs) 

Centre for Reviews and Dissemination at University of York. Freely available - published in 2009

Joanne Briggs Institute (JBI) Manual for Evidence Synthesis Reviewer's Manual. Freely available. Published in 2020.

Campbell Collaboration Training videos.

  1. Question formulation - from 25mins
  2. Searching, coding and quality
  3. Meta Analysis Methods 
  4. Advanced methods
  5. Policy Engagement

Useful     guide on manuals, and reporting guidelines you can use.

You can also use the book, "Doing a systematic review" edited by Boland, A., Cherry, G and Dickson, R  2nd edition (2017). They have a very useful online resource to supplement learning.

Searching Skill toolkit: finding the evidence. 2nd ed. 2014, De Brun. This is available as an ebook Top tips to getting started.

Scoping searches are fairly brief searches of existing literature designed to help you gain an overview of the range and depth of research that exists for a particular research idea. It can cover published work and discover on-going studies. Research proposals are shaped by the results of a scoping search for your research idea.

In short it should ensure that your research complements and adds to existing research so that your proposal is not rejected for ‘re-inventing the wheel’. https://www.rds-yh.nihr.ac.uk/scoping-searchesfunding-upda

Check to make sure no one else has done a Systematic Review on your topic before. Look at PROSPERO and Cochrane Library.

The following websites will give you a comprehensive overview of health related information  - you can filter your results easily to find the specific resources that you need.

A clearly defined, focused review begins with well framed questions. These questions will guide many aspects of the review process such as eligibility, data collection as well as the basis for the search criteria.

Campbell Collaboration Training videos on Question Formulation  - from 25 mins onwards.

Look at chapter on formulating searchable questions from Searching Skills Toolkit: Finding the evidence 2nd ed. 2014 De Brun

Using PICO to formulate a search strategy - link to useful guide

SPICE (Setting, Perspective, Intervention, Comparison, Evaluation

ECLIPSE (Expectation, Client Gp, Location, Impact, Professionals. SErvice)

COPES (Client Orientated Evidence Search (questions come directly from practice)

SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type)

You will need to write a protocol for your systematic review. It should describe the rationale, hypothesis, and planned methods of the review. 

There is detailed guidance on this in PRISMA as well as from the Centre for Reviews and Dissemination at York 

You can then publish your Protocol in Prospero if appropriate. "PROSPERO is an international database of prospectively registered systematic reviews in health and social care, welfare, public health, education, crime, justice, and international development, where there is a health related outcome"

Searching the literature

Please find below the Medical and Health Databases available to you.  A study in 2017 that looked at the optimal combination of databases within a Systematic review, concluded that you should include Embase, MEDLINE (including Epub ahead of print), Web of Science Core Collection, and Google Scholar.

PsycINFO and CINAHL which are more specialized could be used additionally where appropriate. Psychology Doctoral students should use PsycINFO instead of Embase unless their topic is particlularly drug related. 

Please look at this very useful guide on useful tips searching across common database platforms  

Boolean operator

OR      combine synonyms together

cat OR mouse will retrieve all records that contain the word cat plus all the records that contain mouse

Using OR gets MORE

AND    combine different concepts together 

cat AND mouse will retrieve all records that contain both the word cat and the word mouse

NOT   eliminate a word from the results   **use carefully!**

cat NOT mouse will retrieve all records that contain cat, but not any records that also contain mouse

We recommend that you start with OR and build up your search for synonyms for each of your core topics - cat OR feline OR felis

Use AND to combine the results of the synonym searches

Please see Video for more information.

 

The building block of a search strategy are your keywords, these can be "Natural" or "Academic"; a good example of this is Teenager OR adolescent. A good systematic review will have a comprehensive keyword search at its heart where all the appropriate synonyms of a keyword have been used.

Teen OR Teens OR teenager OR teenagers OR Young person OR young adult OR adolescent OR adolescence.

You can then use Syntax (instructions to database) to modify and improve  - example below is using EBSCO platform syntax.

This then becomes Teen* OR (Young N (person* OR people OR adult*)) OR adolescen*

There is a tool to help you with finding synonyms called Search Blocks. You can also use a Thesaurus.

For other database platforms using different syntax, check this guide.

Wildcards and Truncation

These are symbols such as * $ ? that are used to replace a single letter, a missing letter or a series of letters in a word.  When the symbol is placed in the word it is a wildcard and when it is at the end of the word it is a truncation mark. 

Symbols vary according to the database you are using so it is important to check the Help section in each database before you start. Please see this guide to the search operators for different databases (syntax) - truncation, boolean, phrase searching and proximity.

A wildcard is useful to:

  • replace a letter that may be different - ​sterili$ation will find sterilisation OR sterilization
  • a letter that can be missing - colo?r will find colour OR colour

Using a * will search for the main root of the word and any variable endings…

Assess* will find articles containing assess, assessing, assessed, assessment

Please see the videos on this page to help you understand these concepts.

When you enter two words next to each other in a search, most databases will add a hidden  AND when carrying out a search.  This means that they will look for both words anywhere in the record.

For example hand washing will find articles with Hand in the title and Washing in the abstract as well as as Phrase. 

If you want to only search for the words as a phrase then you need to use the right syntax to tell the database what you want.  Usually this is where you add quotation marks around the words.  For example, "Hand washing" or "critical care"

In Scopus use of double quotation marks are used to search for a loose / approximate phrase. To search for an exact phrase you should enclose the phrase in braces: {oyster toadfish}.

If you are searching for a phrase, check the database Help to confirm what you should use.

"Proximity" or "adjacency" can be a very useful tool. It is a way to search for two or more words that occur within a certain number of words from each other. This can be very helpful when your topic can be described in different ways.  For example, 'climate change' could also appear in a paper as 'change in the climate'.  

For example in the Web of Science Core Collection you use NEAR/where is the maximum number of words that separate the terms from each other so to search for 'climate change' or 'change in the climate'.  You would use "climate NEAR/3 change"

Other databases use different terms for this. Check this guide 

EBSCO uses N3 - Neonat* N3 Care will find Neonatal care as well as Care of the Neonate 

Where to find Systematic Reviews

Joanna Briggs Institute. JBI publishes an Evidence based database of Systematic Reviews and Implementation Reports

Campbell Collaboration. International Research Network that promotes positive social and economic change through the use of systematic reviews and evidence synthesis for evidence-based policy and practice 

EPPI-Centre 

The Evidence for Policy and Practice Information and Co-ordinating Centre is a specialist centre for developing methods for:

  • for systematic reviewing and synthesis of research evidence; and
  • for the study of the use research. 

Guideline and Protocols to help you

FINER The characteristics of a good research question are that it be:

Feasible,

Interesting,

Novel,

Ethical, 

Relevant 

Link to Book Chapter PDF

Don't get reporting guidelines and critical appraisal tools confused. See recent article from Logullo et all (2020) for clarification.

Enhancing the QUAlity and Transparency Of health Research - Equator Network is an international initiative to improve the reliability and value of published heath research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines. They have 36 reporting guidelines for Systematic Reviews/Meta Analysis alone! Well worth a look.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.

A PRISMA Flow Diagram, described in the PRISMA Statement is a graphical representation of the flow of citations reviewed in the course of a Systematic Review.

PRISMA P has been produced to help with Protocols

PRISMA-ScR The checklist contains 20 essential reporting items and 2 optional items to include when completing a scoping review 

RepOrting standards for Systematic Evidence Syntheses in environmental research.

ROSES is a set of detailed state-of-the art forms for ensuring evidence syntheses report their methods to the highest possible standards.

Peer Review of Electronic Search Strategies  - PRESS.  The PRESS guideline provides a set of recommendations concerning the information that should be used by librarians and other information specialists when they are asked to evaluate these electronic search strategies.

The full guideline statement and checklist document may be accessed via the following open access article from the Journal of Clinical Epidemiology: ​McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40-6. 

A Measurement Tool to Assess systematic Reviews. AMSTAR seeks to create valid, reliable and useful instruments to help users differentiate between systematic reviews, focusing on their methodological quality and expert consensus as well as facilitate the development of high-quality reviews. Please access Checklist here, and publication list here 

ROBIS (Risk oBias iSystematic Reviews) is a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies). ROBIS has been developed using rigorous methodology and is currently aimed at four broad categories of reviews mainly within healthcare settings: interventions, diagnosis, prognosis and aetiology. There are other tools you can use that might be more appropriate. Please see useful overview here from NHMRC in Australia. 

Boolean operators - refresher

Wild cards and Truncation

Equally important - Don't forget

Evidence that almost half of references in reports on new and emerging nondrug health technologies are grey literature.

Examples of grey literature (see 1.1 for definition) include: conference abstracts, presentations, proceedings; regulatory data; unpublished trial data; government publications; reports (such as white papers, working papers, internal documentation); dissertations/theses; patents; and policies and procedures.

Please see this useful guide from the UK Health Security Agency (formerly Public Health England) which is a very comprehensive list of all sources. Also, useful guide from Leeds University 

Useful chapter on Patient Information Sources from Searching Skills Toolkit: Finding the Evidence 2nd ed. 2014, De Brun

We suggest you use Trip (Turning Research into Practice) which you can filter to Patient information leaflets, Guidelines, or Trials for instance. For Thesis, look at  Proquest Dissertation and Thesis and select "Anywhere except Full Text" and Ethos for UK Thesis, then look at the guides above and choose the most appropriate resources for your subject.

There is usually some time between conducting your initial searches and then writing up your results. You can create alerts with the help of the databases, publisher platforms as well as specific journals.

If you want to update your searches in the databases that you have used to make sure you have found any new articles since you last ran them, please see this useful guide from the University of Alberta. 

 Very good guide on creating alerts for your topic to help you stay up to date.

To help you balance precision against recall, you can use a search filter/hedge to help you.

Please look at this guide to help you find any appropriate filters for study design, age groups, subjects etc. 

Please see this useful guide on various Screening tools you can use. Rayaan is a free tool and McGill have a comprehensive guide on how to use it. 

There is also a tool called Systematic Review Accelerator which you can use to help you de-duplicate and look at your Word Frequency (to help you construct and refine)

"Critical appraisal is the course of action for watchfully and systematically examining research to assess its reliability, value, and relevance in order to direct professionals in their vital clinical decision making." 

Risk of Bias

Understanding how to assess and critically appraise published research to identify potential sources of bias is an essential skill.

ROBIS is a new tool for assessing the risk of bias in systematic reviews. There are other tools that might be more appropriate, please see this useful guide from NHMRC in Australia. 

Hand and Citation Searching (Snowballing or Pearl Growing)

Hand searching is an important element in a Systematic Review. It can involve scanning manually the table of contents and entire issues of key journals and conference proceedings in the subject area you are researching. 

Citation searching, sometimes also referred to as Snowball method, involves reviewing the list of references in a paper and searching for other more recent papers that have cited it since it has been published.

The following resources may be of assistance

Please see "Citation pearl searching" chapter 11 "of Searching Skills Toolkit: Finding the Evidence, 2nd ed. 2014, De Brun and also article by Ruth Hadfield explaining how to use PubMed to help you.

There is new tool called Connected Papers which is a simple network analysis based on similarities between papers. More info here on how it works.

Please see links below for help and guidance on using the Web of Science (WoS)

  • Quick reference guide to get started 
  • Video from Clarivate (who own the WoS) Better searching - truncation, phrase searching and Boolean (5.12)
  • Need more - Go to Clarivate help page  - scroll down to "Strategies for getting results" and click on + sign to open out page of useful videos.