Skip to Main Content

Systematic and complex literature reviews

Tips for undertaking systematic-type reviews

Conduct search

Conduct the literature search

  • The inclusion and exclusion criteria will guide the search strategy
  • The search terms should have been tested for relevancy in the development of the protocol
  • The protocol will have included the search terms, search strategy, and names of the databases
  • The search strategy should be run across multiple databases as coverage and focus of databases will vary
  • Try to run the searches on the same day or as close as possible

Databases - heading box

Databases

Databases

Choosing databases

  • Read systematic reviews on similar topics to see which databases were chosen
  • Consult with your supervisor and the research librarian
  • Read the technical supplement to the Cochrane Handbook linked below for health-related database suggestions
  • Review the list of 28 widely used academic search systems compared in the Gusenbauer and Haddaway article below 

Database tips

  • Use Advanced search options if available.
  • Create an account on each database that you use, so that you can save searches and return to edit them if necessary.
  • Search syntax may vary between databases – look in the search help that each database provides.
  • Default search fields and subject terms vary, so you will need to adjust your search between databases.
  • Start broad and filter down.
  • Search for each concept then use the Search History (if available) to combine your concepts. This allows you to change the concepts you are combining to make an effective search.

Common database conventions

Check the help link on your database for more guidance.

* (asterisk)

The asterisk is a truncation character. Substitutes for word endings, For example:

adolescen* will find adolescent, adolescents, adolescence

" " (quotation marks)

Use quotation marks around phrases (2 or more words), to force a search for that exact phrase:

"mental health" will find these two words, in this exact order

Field searching Use dropdown boxes to choose the fields in which you want to search for your keywords, rather than the whole record. Conventions are to search the Title, Abstract, and Keyword fields (in addition to any Subject Heading fields).
Subject headings

Can also be called Thesaurus terms, MeSH (Medical Subject Headings), or Index terms. These are a controlled vocabulary where articles are assigned to different subject headings. They can differ between databases. For example:

In PubMed, the MeSH heading for used for teenagers is Adolescent, in another database the CINAHL subject heading is Adolescence.

 

Search strategy - heading box

Search strategy

Search strategy

Search strategy

Use the concepts developed in your research framework (eg PICO) to help construct the search strategy.

  • Download the blank search strategy Word document linked below to record your keywords and synonyms for each concept. This preparation will help you develop a search string that you will modify to suit the various databases that you search.
  • Note that keywords will OR within a concept, and your final search will AND across all the concepts.
  • Check the Help files for each database. Search for terms such as Advanced, Boolean, wildcards, or truncation to check the required format.
  •  For example, in most databases the truncation character (asterisk *) is used to substitute for word endings. Most databases use quotation marks for phrases. Some databases (but not all) recognise the wildcard character (?) to replace letters.

Example search strategy document

See below for a preliminary search strategy document for the proposed Scoping review on the use of CBT in management of anxiety in teenagers experiencing insomnia. Note the use of quotation marks around phrases, and the truncation character * to substitute for word endings.

Keeping track of your searches

Systematic type of reviews need to be reproducible and have their process transparent. Keeping track of your searches and the results is essential. A search log can be maintained using tools such as Excel. You can show the evolution of your search, and allows you to review the search strategy, and to expand or refine your search terms to make the search more effective. It also helps when re-creating (or translating) the search to work in different databases to have the search terms ready to copy and paste.

Keep accurate details of every search – record details such as:

  • date of search

  • search strategy exactly how it was typed in the database

  • name of the database searched (or platform and multiple databases)

  • any field restrictions (eg search conducted in Title-Abstract-Keyword fields)

  • notes

  • whether search has been saved, or an alert set up (y/n + date)

  • what login was used for the database (so that the search can be retrieved and re-run at a later date)

Reference management software such as EndNote can be utilised to organise search results, and record numbers of results.

Click on the image below to download a sample search log with the type of information you can record. You can also download the blank search log to record your searches.

Image thumbnail of search log

Subject headings

Some databases offer subject headings. The term each database uses might differ - they might be called MeSH headings, CINAHL headings, or index terms. These are controlled vocabularly terms that are used to index the articles in the database. If a database offers subject headings, you will want to combine subject heading searches with your keyword searches. Check the indexing of other key articles to see what subject headings were used. You can then OR these subject heading searches with your keyword searches for each concept in your search strategy.

View the video from John Hopkins University below on building a PubMed search with MeSH headings and keywords. Links to tools to help identify MeSH headings are below the video.

Sensitivity vs precision

It takes time to craft a search that is balanced between sensitivity and precision. Sensitivity retrieves most of the relevant results but also means a great number will need to be screened and rejected. Precision reduces the number of results to screen, but risks missing some relevant results.

The Cochrane Handbook recommends to  "maximize sensitivity whilst striving for reasonable precision" (Cochrane, Chapter 4). See the link below for a discussion on sensitivity, and their estimate of screening database results to be between 60 to 120 per hour.

Clipart of hourglass and clockSearching takes time

This is an experimental stage where you are perfecting the search. You might amend your search strategy document with new keywords or adding subject headings (eg MeSH headings in PubMed). Once you are happy with the search, you can then translate it for the other databases, since they may have different subject terms and keyword fields. Use the Search Log document (on the Keeping track tab above) to record your trial and final search strings.

Time management image from Freesvg.org. Public Domain.

Exporting results

Export to EndNote

Create logins for each database, and once happy with your search, save your final search string. You can then run all your saved searches and export results to EndNote.

You will create a group for each set of results in your EndNote library. Methods of exporting large result sets from databases will vary. See the Monash EndNote guide below for tips on bulk export of results from different databases.

Change the name of your database groups in EndNote by adding the total number of records retrieved from each database. This will help you record the original figures retrieved in your PRISMA flowchart later on. Remember removing duplicates will change these figures, so you will want to have a record of those original numbers.

Remove duplicates using the EndNote > Find Duplicates tool, and by manual deletion. Record these figures for your PRISMA flowchart.