Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Optimizing literature search in systematic reviews – are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders?

BMC Medical Research Methodology volume 16, Article number: 161 (2016) Cite this article

Abstract

Background

When conducting systematic reviews, it is essential to perform a comprehensive literature search to identify all published studies relevant to the specific research question. The Cochrane Collaborations Methodological Expectations of Cochrane Intervention Reviews (MECIR) guidelines state that searching MEDLINE, EMBASE and CENTRAL should be considered mandatory. The aim of this study was to evaluate the MECIR recommendations to use MEDLINE, EMBASE and CENTRAL combined, and examine the yield of using these to find randomized controlled trials (RCTs) within the area of musculoskeletal disorders.

Methods

Data sources were systematic reviews published by the Cochrane Musculoskeletal Review Group, including at least five RCTs, reporting a search history, searching MEDLINE, EMBASE, CENTRAL, and adding reference- and hand-searching. Additional databases were deemed eligible if they indexed RCTs, were in English and used in more than three of the systematic reviews. Relative recall was calculated as the number of studies identified by the literature search divided by the number of eligible studies i.e. included studies in the individual systematic reviews. Finally, cumulative median recall was calculated for MEDLINE, EMBASE and CENTRAL combined followed by the databases yielding additional studies.

Results

Deemed eligible was twenty-three systematic reviews and the databases included other than MEDLINE, EMBASE and CENTRAL was AMED, CINAHL, HealthSTAR, MANTIS, OT-Seeker, PEDro, PsychINFO, SCOPUS, SportDISCUS and Web of Science. Cumulative median recall for combined searching in MEDLINE, EMBASE and CENTRAL was 88.9% and increased to 90.9% when adding 10 additional databases.

Conclusion

Searching MEDLINE, EMBASE and CENTRAL was not sufficient for identifying all effect studies on musculoskeletal disorders, but additional ten databases did only increase the median recall by 2%. It is possible that searching databases is not sufficient to identify all relevant references, and that reviewers must rely upon additional sources in their literature search. However further research is needed.

Peer Review reports

Background

Systematic reviews (SR) and meta-analyses (MA) are key elements in both evidence-based healthcare [1] and evidence-based research [2] By synthesizing the available evidence, SRs support clinicians in making well-informed decisions about health care [3] and researchers in deciding which topics are the most relevant for new research [4]. When conducting SRs, it is essential to perform a comprehensive literature search to identify all published studies relevant to the research question as a failure to do so can result in selection bias and distort the conclusion of the review by potentially over- or underestimating of the treatment effect [3, 5]. One of the recommended methods to identify scientific literature in health science is searching electronic databases [3, 6]. However, when doing a high quality search two main questions arise; which databases is necessary to searched, and how many? According to the Cochrane Collaborations Methodological Expectations of Cochrane Intervention Reviews (MECIR) three main databases: MEDLINE, EMBASE and CENTRAL are mandatory electronic databases to search when performing a Cochrane Review [7]. Several studies indicate that searching MEDLINE identifies the highest number of studies [8–10] and others that the gains from searching beyond MEDLINE and in particular EMBASE are modest [11], however multiple studies have found that searching MEDLINE alone is not sufficient [8–10, 12–19]. In addition, when analysing the use of databases in Cochrane reviews, it was found that between 1 and 27 different databases was used [20], even though some studies indicates that searching no more than 3–5 databases seems to be sufficient,[8, 16, 17] and searching only one database would not be enough [19]. Even though MEDLINE, EMBASE and other major medical databases yield a high proportion of relevant studies, some studies found it necessary to include other sources such as reference- and citation search, browsing conference proceedings, asking experts and alike to identify all the relevant studies [9, 14, 17, 19]. The difference between the results from these studies could be due to their evaluation of different areas or due to the methods used to search the different databases; hence some of the above mentioned studies construct a new search strategy to identify studies in a given area thereby making the evaluation be dependent on not just the database, but also the quality/accuracy of the search strategy constructed [21–23]. The great variations thus indicate a need to evaluate if the MECIR guideline recommendations to search MEDLINE, EMBASE and CENTRAL combined would be enough when performing a literature search or whether additional databases should be added to this list.

In order to focus this evaluation, we choose to concentrate on musculoskeletal disorders. The area of musculoskeletal injuries and diseases is the leading cause of long-term pain and physical disability [24–26] and are associated with 130 million health care encounters and estimated to cost over $50 billion annually in the United States [27]. In addition, the Cochrane Musculoskeletal Review Group (CMSG) is among the largest review groups in the Cochrane Collaboration, responsible for more than 200 SRs.

The aim of this study is therefore to evaluate the relative recall in the databases recommended by MECIR for systematic literature searches within the area of musculoskeletal disorders. Furthermore, this study addressed the question: What is the increase in recall when searching additional databases to searching MEDLINE, EMBASE and CENTRAL combined?

Methods

Selection of systematic reviews

All SRs from the Cochrane Database of Systematic Reviews (CDSR) published by CMSG were obtained [28]. SRs were excluded if they: (i) did not include at least five randomised controlled trials (RCT), as the consequence of missing one study in reviews with few studies included would affect the overall percentage more than with a higher total of included studies. (ii) had been withdrawn, (ii) did not report any search history in the SR (iii) did not search all of the following sources: MEDLINE, EMBASE, CENTRAL, reference- and hand searching, as recommended by The Cochrane Handbook [3] and MECIR guidelines [7]. This strategy was defined in order to identify systematic reviews, which had included all (or close to all) relevant studies related to at certain research question by both searching electronic databases and other sources.

The recall was used to evaluate the ability of a search strategy to identify all relevant studies [29]. Recall can be defined as the percentage of relevant records retrieved divided by the total number of included studies in the individual systematic reviews. However, to estimate true recall one need to know the total amount of relevant records in a database, which is not an easy (if not impossible) task. Thus often, relative recall is estimated by firstly defining a pool of relevant records (the included studies in a SR) and then determines what proportion of this pool the literature search retrieves.[30, 31] In this study we therefore used the included studies in each of the included SRs as the pool of relevant records.

Identification of databases

From the pool of SRs, a list of databases used was created. Databases were ranked in descending order according to how many SRs that had searched the database. Databases other than MEDLINE, EMBASE and CENTRAL were deemed eligible if they (i) was indexing RCTs, (ii) was in English, (iii) was used by at least three SRs.

Data-extraction

The following data were extracted for each included SR: (i) details of the search strategy as described in the review (ii) date of when the search was performed or updated (iii) full bibliographic details of all primary studies included in the SR (i.e. title of the study, author names, journal title, publications year etc.).

Searching individual databases

The search strategy/strategies reported in each SR was replicated and used for searching all the databases included in the SR. For databases with no reported search strategy, the MEDLINE search strategy was replicated and searched in all included databases. MEDLINE syntax (i.e. fields, truncation, adjacency) was modified to the individual database. When possible, the exact search dates from the SR was used for each database. However, CENTRAL for instance, only allows specification by month and year. End Note ×7.5.3 software (Thomson Reuters™) was used to manage records retrieved from searches of electronic databases.

Statistical analysis

Relative recall for each of the included database was calculated separately and for each of the included SR. Relative median recall for each database was calculated for all included SRs combined. Cumulative median relative recall was estimated for searches in MEDLINE, EMBASE and CENTRAL adding databases in descending order (based on how often the databases was searched in the SRs published by CMSG). Data managing was performed using Microsoft Excel 2016 and data analysis was performed using STATA version 13.1 (StataCorp, College Station, Texas) software package.

Additional analyses

Subgroup analysis was pre-specified and planned to assess the cumulative median recall for subgroups, rehabilitation, medicine or other content (surgery, lifestyle intervention, electrical stimulation etc.) as previous studies have found differences in recommendations depending on the topics searched [32, 33]. One post-hoc sensitivity analysis was conducted to address to what extend the inclusion of SRs with a cut-off of three included RCTs instead of five would affect the result.

Results

Eligible databases and systematic reviews

A timeline is displayed in Fig. 1. A set of 164 SRs where identified and obtained from the CMSG on March 3, 2013 and revisited for an update on July 3 2013 by the first author (Fig. 2). Of the 164 SRs assessed for eligibility by title and abstract, 10 were excluded. Nine for being withdrawn and one for being an overview of reviews. Of the remaining 154 SRs assessed in full-text, 114 were excluded, as they did not search one or more of the following sources: MEDLINE-, CENTRAL-, EMBASE, or reference- and hand searching. 11 SRs were excluded, as they did not include five or more studies in their analysis. Six were excluded for not reporting any search strategy in the SR, neither reporting where one could be acquired. A final set of 23 SRs finally met all inclusion criteria [34–56], (Table 1).

Fig. 1
figure 1

A timeline of the selecting, inclusion and analysis process

Full size image
Fig. 2
figure 2

Flowchart for inclusion of Cochrane reviews and databases

Full size image
Table 1 Characteristics of the included Cochrane reviews
Full size table

The generated list of databases other than MEDLINE, EMBASE and CENTRAL included a total of 58 databases identified in the 23 included SRs. Of these 58 databases, 48 databases were excluded; 10 did not index RCTs (i.e. trial registry etc.), nine where included in other databases (i.e. Premedline in MEDLINE etc.), 28 were used in less than three SRs, and one database was not in English. The following 10 databases met the inclusion criteria: AMED (via EBSCOhost), CINAHL (via EBSCOhost), HealthSTAR (via OVID), MANTIS (via OVID), OT-Seeker, PEDro, PsychINFO (via OVID), SCOPUS, SportDISCUS (via EBSCOhost) and Web of Science. Searching MEDLINE was performed using the host specified in the SR (i.e. via OVID or Pubmed), EMBASE via the OVID and CENTRAL via the Wiley InterScience portal.

Characteristics of the included systematic reviews

The 23 SRs included a total of 365 primary studies. Each review included from 5 to 103 studies (median 10) (Table 1). The number of search strategies reported in the SRs ranged from 1 to 7 (median 2). Eleven out of 23 SRs reported only one search strategy; 4 of which reported a “standard search strategy” that was adapted to other databases searched, while 7 reported the search strategy used for MEDLINE (Table 1). Of the 23 SRs, the intervention in 5 was classified as rehabilitation, 14 as medicine and 4 was classified as other content (Table 1).

Synthesis of results

Table 2 displays the median relative recall for the combined search in MEDLINE, EMBASE and CENTRAL and for the additional 10 databases included. Data shown are the median recall and interquartile range (IQR) from the total number of SRs and for the three subgroups separately. MEDLINE, EMBASE and CENTRAL combined yielded a median recall of 88.9% (IQR 81.6–100%), followed by SCOPUS (85.7%) and HealthSTAR (83.3%) (Table 2).

Table 2 Median recall analysis for each database in descending order
Full size table

Results of the overall cumulative median analysis on relative recall are displayed in Fig. 3 and Table 3. The most exhaustive search (i.e. the minimum number of databases required to be searched to retrieve the maximum number of studies) involved searching MEDLINE, EMBASE and CENTRAL with the addition of SCOPUS and CINAHL. Results show that adding these databases to MEDLINE, EMBASE and CENTRAL increased the median recall by 2.0 percentage points, from 88.9% to a median recall of 90.9% (IQR 83.3–100%). Adding the remaining 8 databases did not increase the median recall.

Fig. 3
figure 3

The accumulating percentage as a boxplot

Full size image
Table 3 Overall cumulative median analysis on relative recall
Full size table

Additional analyses

Subgroup analysis

Subgroup analyses according to content area demonstrated some variations. The most exhaustive search for the rehabilitation group involved searching MEDLINE, EMBASE and CENTRAL with the addition of SCOPUS and CINAHL for a cumulative median recall of 100% (IQR 60–100%) (Table 4), the medicine group with the addition of SCOPUS for a cumulative median recall of 87.3% (IQR 83.3–97.7%) (Table 4), and the other content groups with the addition of Scopus and CINAHL for a cumulative median recall of 100% (IQR 97.7–100%) (Table 4).

Table 4 Cumulative subgroup analysis according to content area
Full size table

Post-hoc analysis

Post-hoc sensitivity analysis showed that with the inclusion of SRs with at least three included RCTs, 4 SRs would be added to the analyses [57–60]. The analyses showed that the cumulative median recall increased when adding these SRs, however the IQR remained unchanged (Table 5).

Table 5 Overall cumulative median analysis on relative recall with a cut-off of three included studies
Full size table

Discussion

This study supports the recommendations by Cochrane Collaboration to prioritize MEDLINE, EMBASE and CENTRAL as the basic databases for literature search to locate RCTs in the musculoskeletal area. Secondly, this study indicates that besides MEDLINE, EMBASE and CENTRAL a literature search to locate RCTs in the musculoskeletal area could also consider SCOPUS and CINAHL. Finally, this study indicates that even with the addition search of 10 other often used databases median recall is not improved noticeably.

Thirteen different databases were not enough to identify all relevant references. Searching MEDLINE, EMBASE and CENTRAL retrieved 88.9%, and searches in 10 additional databases increased the median recall by only 2 percentage point. Thus, results from this study could be interpreted, as an indication that searching databases is not sufficient to identify all relevant references and that other sources must be included in the literature search in order to achieve a larger recall. This study does not evaluate which source that may be the most important. Savoie et al. and Helmer et al. [61, 62] found that 29.2% of all items retrieved for two SRs could be uncovered by extended systematic search methods; searching subject- specific or specialized databases (including trial registries), hand-searching, scanning reference lists, and communicating personally with experts. Yet Robinson et al. [63] recently showed that researchers do not cite all possible previous trials, and that less than half (38%) of RCTs could be identified by citation network searching.

It therefore remains to be evaluated how much higher recall could be achieved by supplementing the database search with reference and/or citation search, and which impact if any these additional sources have on the pooled estimate effect.

Searching SCOPUS and CINAHL increased the median recall by 2 point. Yet, as results from the subgroup analysis showed, each database contributed differently depending on the field groups searched. SCOPUS increased the median recall slightly in the Other group, and had some effect on the IQR in all three groups. This could be due to the fact that SCOPUS is a generic database containing studies from a wide range of subject fields. The large increase in median recall in the rehabilitation group searching CINAHL could be because CINAHL is a database including research from health care professionals often involved with rehabilitation. The fact that CINAHL only increased the recall in the area of rehabilitation are supported by Beckles et al. who strongly recommends that the database should be relegated too selective rather than routine searching due to a very low proportion of unique references [64].

Results from the post-hoc sensitivity analyses showed not surprisingly that the inclusion of studies with a low proportion of included studies could introduce high risk of bias of the results. Adding only four more studies increased the median recall to 100% and by 10% compared to the main results. Yet, as the IQR of the results are unchanged, this reinforce the notion, that searching additional databases is less likely to add more studies.

To our knowledge only one earlier study concluded that one database was enough in order to achieve full recall. Kelly et al. [65] concluded that MEDLINE was enough to identify all relevant studies for their specific question. However, they concluded that to fully capture the complete body of available literature on other subjects might require searching multiple databases [20]. This is strongly supported by a number of other studies [8–13, 15, 16, 18–22, 33, 66–78]. Studies evaluating this question within the musculoskeletal field make the same conclusion: searching more than one database is necessary [14, 17, 23, 32, 79–81]. Based on results from earlier studies and the results from the present study, recommendation for an optimal systematic literature search to locate RCTs within the musculoskeletal area may be to use the three generic databases: MEDLINE; EMBASE and CENTRAL, and an additional two or more other databases. However, this search should ad other sources such as reference- and citation search, grey literature, conference proceedings, and contact experts within the area as results from this study suggest that 10% could be missed when only searching electronical databases to identify relevant studies.

Limitations of this study

This study has some limitations. An important limitation of this study and other studies evaluating the recall of a systematic literature search is the definition of the true number of studies that should be identified. In this study we defined this as the number of studies deemed relevant (i.e. included) in a SR, yet as a SR seek to answer a well-defined question, there are some limitations to whether the included SRs in this study fully represent the various areas of the musculoskeletal field. Another limitation of any study reproducing an original search strategy at a later date is that the contents and indexing of databases change over time, and not all of these changes can be rewound.

Another limitation to this study is the underlining assumption that the systematic literature search strategy used in each SR did capture all relevant studies in the database searched. The result from the present study does not evaluate this question, yet Sampson et al. [82] found that errors in electronic search strategies reduce the effectiveness of electronic search strategies. Further research is needed to evaluate not only the recall of studies retrieved using a search strategy, but also comparing this to the recall of studies indexed in a database by bibliographic verification: searching for known items [83], thereby addressing the key question, what is indexed in a database? and what is actually retrieved when searching this database?

The aim of this study was to evaluate the relative recall in the databases recommended by MECIR for systematic literature searches to locate RCTs within the area of musculoskeletal disorders. The use and limitations of the method to the musculoskeletal area thus clearly limits the general conclusion from this study. However, our results are in line with other studies evaluating literature search in electronic databases.

The strengths of this study lies in the systematic approach of selecting Cochrane SRs of the highest quality, and combining results of the literature search from these SRs in a way that make SRs with a high number of studies included equal to those with low number of studies included. This is also one of few studies [20, 23, 84] that have combined multiple databases using cumulative analysis, thereby accepting what researchers have urged in the past, that searching one database is not enough but investigating what a combined search would yield.

Conclusions

Searching MEDLINE, EMBASE and CENTRAL is not sufficient for identifying all effect studies within the area of musculoskeletal disorders. Literature searches in ten additional databases only increases the median recall by 2 percentage point.

It remains to be evaluated how much higher the relative recall could be achieved by supplementing the database search with reference and citation search. Further studies where the same methods are applied on different content areas needs to be performed, to see if the assumption that the way to perform a search depends on the content area is true or not. It is possible that searching databases is not sufficient to identify all relevant references, and that reviewers must rely upon additional sources in their literature search, but further research on these additional sources is needed.

Abbreviations

CDSR:

Cochrane database of systematic reviews

CMSG:

Cochrane musculoskeletal review group

IQR:

Interquartile range

M + E + C:

MEDLINE + EMBASE + CENTRAL

M + E + C + S:

MEDLINE + EMBASE + CENTRAL + SCOPUS

M + E + C + S + Ci:

MEDLINE + EMBASE + CENTRAL + SCOPUS+ Cinahl

MA:

Meta-analyses

MECIR:

Methodological expectations of cochrane intervention reviews

RCT:

Randomised controlled trials

SR:

Systematic reviews

References

  1. Gray JAM, Booth A, Booth S, Ison E. Evidence-based healthcare. Edinburgh: Churchill Livingstone; 2001.

    Google Scholar 

  2. Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, Jamtvedt G, Nortvedt MW, Christensen R, Chalmers I. Towards evidence based research. BMJ. 2016;355:i5440.

    Article  PubMed  Google Scholar 

  3. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley & Sons; 2008.

    Book  Google Scholar 

  4. Chalmers I, Nylenna M. A new network to promote evidence-based research. Lancet. 2014;384(9958):1903–4.

    Article  PubMed  Google Scholar 

  5. Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D, Falck-Ytter Y, et al. GRADE guidelines: 5. Rating the quality of evidence--publication bias. J Clin Epidemiol. 2011;64(12):1277–82.

    Article  PubMed  Google Scholar 

  6. Klassen TP, Jadad AR, Moher D. Guides for reading and interpreting systematic reviews: I. Getting started. Arch Pediatr Adolesc Med. 1998;152(7):700–4.

    Article  CAS  PubMed  Google Scholar 

  7. Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological standards for the conduct of new Cochrane Intervention Reviews. The Cochrane Unit. 2013;2:3.

    Google Scholar 

  8. Bayliss SE, Davenport CF, Pennant ME. Where and how to search for information on the effectiveness of public health interventions--a case study for prevention of cardiovascular disease. Health Info Libr J. 2014;31(4):303–13.

    Article  PubMed  Google Scholar 

  9. Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: Added value or diminishing returns? Complement Ther Med. 2004;12(4):228–32.

    Article  CAS  PubMed  Google Scholar 

  10. Rollin L, Darmoni S, Caillard JF, Gehanno JF. Searching for high-quality articles about intervention studies in occupational health - What is really missed when using only the Medline database? Scand J Work Environ Health. 2010;36(6):484–7.

    Article  PubMed  Google Scholar 

  11. Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68(9):1076–84.

    Article  PubMed  Google Scholar 

  12. Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Inj Prev. 2008;14(6):401–4.

    Article  CAS  PubMed  Google Scholar 

  13. Minozzi S, Pistotti V, Forni M. Searching for rehabilitation articles on MEDLINE and EMBASE. An example with cross-over design. Arch Phys Med Rehabil. 2000;81(6):720–2.

    Article  CAS  PubMed  Google Scholar 

  14. Murphy LS, Reinsch S, Najm WI, Dickerson VM, Seffinger MA, Adams A, Mishra SI. Spinal palpation: The challenges of information retrieval using available databases. J Manipulative Physiol Ther. 2003;26(6):374–82.

    Article  PubMed  Google Scholar 

  15. Sampson M, Cogo E, Ajiferuke I, Manheimer E, Campbell K, Daniel R, Moher D. Searching for controlled trials of complementary and alternative medicine: A comparison of 15 databases. Evid Based Complement Alternat Med. 2011;2011:858246.

    PubMed  PubMed Central  Google Scholar 

  16. Shariff SZ, Sontrop JM, Iansavichus AV, Haynes RB, Weir MA, Gandhi S, Cuerden MS, Garg AX. Availability of renal literature in six bibliographic databases. Clin Kidney J. 2012;5(6):610–7.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic Databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.

    Article  CAS  PubMed  Google Scholar 

  18. Vickers AJ. Bibliometric analysis of randomized trials in complementary medicine. Complement Ther Med. 1998;6(4):185–9.

    Article  Google Scholar 

  19. Whiting P, Westwood M, Burke M, Sterne J, Glanville J. Systematic reviews of test accuracy should search a range of databases to identify primary studies. J Clin Epidemiol. 2008;61(4):357.

    Article  PubMed  Google Scholar 

  20. Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: Rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.

    Article  PubMed  Google Scholar 

  21. Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.

    Article  PubMed  Google Scholar 

  22. Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Slobogean GP, Verma A, Giustini D, Slobogean BL, Mulpuri K. MEDLINE, EMBASE, and Cochrane index most primary studies but not abstracts included in orthopedic meta-analyses. J Clin Epidemiol. 2009;62(12):1261–7.

    Article  PubMed  Google Scholar 

  24. Woolf AD, Pfleger B. Burden of major musculoskeletal conditions. Bull World Health Organ. 2003;81(9):646–56.

    PubMed  PubMed Central  Google Scholar 

  25. Horton R. GBD 2010: understanding disease, injury, and risk. Lancet. 2012;380(9859):2053–4.

    Article  PubMed  Google Scholar 

  26. HSE: The health and safety executive statistics 2010/11. In.: http://www.hse.gov.uk/statistics/overall/hssh1011.pdf Accessed 31 Jan 2016.

  27. Utterback DF, Schnorr TM: Use of workers’ compensation data for occupational safety and health: proceedings from June 2012 workshop. In US Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. In. http://www.cdc.gov/niosh/topics/workercomp/cwcs/publications.html: Assessed 25 Aug 2016.

  28. Cochrane Musculoskeletal - Our Reviews. In. http://musculoskeletal.cochrane.org/our-reviews: Accessed 3 Jul 2013

  29. Harter SP. Online information retrieval : concepts, principles and techniques. Orlando: Academic; 1986.

    Google Scholar 

  30. Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6:33.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Vincent S, Greenley S, Beaven O. Clinical Evidence diagnosis: Developing a sensitive search strategy to retrieve diagnostic studies on deep vein thrombosis: a pragmatic approach. Health Info Libr J. 2003;20(3):150–9.

    Article  PubMed  Google Scholar 

  32. Michaleff ZA, Costa LOP, Moseley AM, Maher CG, Elkins MR, Herbert RD, Sherrington C. CENTRAL, PEDro, PubMed, and EMBASE are the most comprehensive databases indexing randomized controlled trials of physical therapy interventions. Phys Ther. 2011;91(2):190–7.

    Article  PubMed  Google Scholar 

  33. Watson RJD, Richardson PH. Identifying randomized controlled trials of cognitive therapy for depression: Comparing the efficiency of embase, medline and PsycINFO bibliographic databases. Psychol Psychother Theory Res Pract. 1999;72(4):535–42.

    Google Scholar 

  34. Adie S, Kwan A, Naylor JM, Harris IA, Mittal R. Cryotherapy following total knee replacement. Cochrane Database Syst Rev. 2012;9, CD007911.

    Google Scholar 

  35. Bartels EM, Lund H, Hagen KB, Dagfinrud H, Christensen R, Danneskiold-Samsoe B. Aquatic exercise for the treatment of knee and hip osteoarthritis. Cochrane Database Syst Rev. 2007;4, CD005523.

    Google Scholar 

  36. Bellamy N, Campbell J, Robinson V, Gee T, Bourne R, Wells G. Viscosupplementation for the treatment of osteoarthritis of the knee. Cochrane Database Syst Rev. 2006;2, CD005321.

    Google Scholar 

  37. Bellamy N, Campbell J, Robinson V, Gee T, Bourne R, Wells G. Intraarticular corticosteroid for treatment of osteoarthritis of the knee. Cochrane Database Syst Rev. 2006;2, CD005328.

    Google Scholar 

  38. Coghlan JA, Buchbinder R, Green S, Johnston RV, Bell SN. Surgery for rotator cuff disease. Cochrane Database Syst Rev. 2008;1, CD005619.

    Google Scholar 

  39. Colebatch AN, Marks JL, Edwards CJ. Safety of non-steroidal anti-inflammatory drugs, including aspirin and paracetamol (acetaminophen) in people receiving methotrexate for inflammatory arthritis (rheumatoid arthritis, ankylosing spondylitis, psoriatic arthritis, other spondyloarthritis). Cochrane Database Syst Rev. 2011;11, CD008872.

    Google Scholar 

  40. Cranney A, Welch V, Adachi JD, Homik J, Shea B, Suarez-Almazor ME, Tugwell P, Wells G. Calcitonin for the treatment and prevention of corticosteroid-induced osteoporosis. Cochrane Database Syst Rev. 2000;2, CD001983.

    Google Scholar 

  41. De Morton NA, Keating JL, Jeffs K. Exercise for acutely hospitalised older medical patients. Cochrane Database Syst Rev. 2007;1, CD005955.

    Google Scholar 

  42. Fidelix TS, Soares BG, Trevisani VF. Diacerein for osteoarthritis. Cochrane Database Syst Rev. 2006;1, CD005117.

    Google Scholar 

  43. Karjalainen K, Malmivaara A, Van Tulder M, Roine R, Jauhiainen M, Hurri H, Koes B. Multidisciplinary rehabilitation for fibromyalgia and musculoskeletal pain in working age adults. Cochrane Database Syst Rev. 2000;2, CD001984.

    Google Scholar 

  44. Katchamart W, Trudeau J, Phumethum V, Bombardier C. Methotrexate monotherapy versus methotrexate combination therapy with non-biologic disease modifying anti-rheumatic drugs for rheumatoid arthritis. Cochrane Database Syst Rev. 2010;4, CD008495.

    Google Scholar 

  45. Khan F, Ng L, Gonzalez S, Hale T, Turner-Stokes L. Multidisciplinary rehabilitation programmes following joint replacement at the hip and knee in chronic arthropathy. Cochrane Database Syst Rev. 2008;2, CD004957.

    Google Scholar 

  46. Lethaby A, Lopez-Olivo MA, Maxwell L, Burls A, Tugwell P, Wells GA. Etanercept for the treatment of rheumatoid arthritis. Cochrane Database Syst Rev. 2013;5, CD004525.

    Google Scholar 

  47. Nuesch E, Rutjes AW, Husni E, Welch V, Juni P. Oral or transdermal opioids for osteoarthritis of the knee or hip. Cochrane Database Syst Rev. 2009;4, CD003115.

    Google Scholar 

  48. Osiri M, Shea B, Robinson V, Suarez-Almazor M, Strand V, Tugwell P, Wells G. Leflunomide for treating rheumatoid arthritis. Cochrane Database Syst Rev. 2003;1, CD002047.

    Google Scholar 

  49. Richards BL, Whittle SL, Buchbinder R. Antidepressants for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2011;11, CD008920.

    Google Scholar 

  50. Richards BL, Whittle SL, Buchbinder R. Muscle relaxants for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2012;1, CD008922.

    PubMed  Google Scholar 

  51. Ruiz Garcia V, Jobanputra P, Burls A, Cabello JB, Galvez Munoz JG, Saiz Cuenca ES, Fry-Smith A. Certolizumab pegol (CDP870) for rheumatoid arthritis in adults. Cochrane Database Syst Rev. 2011;2, CD007649.

    Google Scholar 

  52. Rutjes AW, Nuesch E, Sterchi R, Juni P. Therapeutic ultrasound for osteoarthritis of the knee or hip. Cochrane Database Syst Rev. 2010;1, CD003132.

    Google Scholar 

  53. Rutjes AW, Nuesch E, Sterchi R, Kalichman L, Hendriks E, Osiri M, Brosseau L, Reichenbach S, Juni P. Transcutaneous electrostimulation for osteoarthritis of the knee. Cochrane Database Syst Rev. 2009;4, CD002823.

    Google Scholar 

  54. Wajon A, Ada L, Edmunds I. Surgery for thumb (trapeziometacarpal joint) osteoarthritis. Cochrane Database Syst Rev. 2005;4, CD004631.

    Google Scholar 

  55. Whittle SL, Richards BL, Husni E, Buchbinder R. Opioid therapy for treating rheumatoid arthritis pain. Cochrane Database Syst Rev. 2011;11, CD003113.

    Google Scholar 

  56. Winzenberg TM, Powell S, Shaw KA, Jones G. Vitamin D supplementation for improving bone mineral density in children. Cochrane Database Syst Rev. 2010;10, CD006944.

    Google Scholar 

  57. Brosseau L, Judd MG, Marchand S, Robinson VA, Tugwell P, Wells G, Yonge K. Transcutaneous electrical nerve stimulation (TENS) for the treatment of rheumatoid arthritis in the hand. Cochrane Database Syst Rev. 2003;3, CD004377.

    Google Scholar 

  58. O’Donnell S, Cranney A, Wells GA, Adachi JD, Reginster JY. Strontium ranelate for preventing and treating postmenopausal osteoporosis. Cochrane Database Syst Rev. 2006;3, CD005326.

    Google Scholar 

  59. Richards BL, Whittle SL, Buchbinder R. Neuromodulators for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2012;1, CD008921.

    PubMed  Google Scholar 

  60. Rome K, Ashford RL, Evans A. Non-surgical interventions for paediatric pes planus. Cochrane Database Syst Rev. 2010;7, CD006311.

    Google Scholar 

  61. Savoie I, Helmer D, Green CJ, Kazanjian A. Beyond Medline: reducing bias through extended systematic review search. Int J Technol Assess Health Care. 2003;19(1):168–78.

    Article  PubMed  Google Scholar 

  62. Helmer D, Savoie I, Green C, Kazanjian A. Evidence-based practice: extending the search to find material for the systematic review. Bull Med Libr Assoc. 2001;89(4):346–52.

    CAS  PubMed  PubMed Central  Google Scholar 

  63. Robinson KA, Dunn AG, Tsafnat G, Glasziou P. Citation networks of related trials are often disconnected: implications for bidirectional citation searches. J Clin Epidemiol. 2014;67(7):793–9.

    Article  PubMed  Google Scholar 

  64. Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, Alderson P. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.

    Article  PubMed  Google Scholar 

  65. Kelly L, St Pierre-Hansen N. So many databases, such little clarity: Searching the literature for the topic aboriginal. Can Fam Physician. 2008;54(11):1572–1573.e1575.

    PubMed  PubMed Central  Google Scholar 

  66. Adams CE, Power A, Frederick K, Lefebvre C. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.

    Article  CAS  PubMed  Google Scholar 

  67. Aker PD, McDermaid C, Opitz BG, White MW. Searching chiropractic literature: A comparison of three computerized databases. J Manip Physiol Ther. 1996;19(8):518–24.

    CAS  Google Scholar 

  68. Crumley ET, Wiebe N, Cramer K, Klassen TP, Hartling L. Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review. BMC Med Res Methodol. 2005;5:24.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309(6964):1286–91.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Haafkens J, Moerman C, Schuring M, Van Dijk F. Searching bibliographic databases for literature on chronic disease and work participation. Occup Med. 2006;56(1):39–45.

    Article  Google Scholar 

  71. Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, Yao L, Yang K. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

    Article  PubMed  Google Scholar 

  72. Lorenzetti DL, Topfer LA, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

    Article  PubMed  Google Scholar 

  73. Lovarini M, Wallen M, Imms C. Searching for evidence in pediatric occupational therapy using free versus subscription databases: A comparison of outcomes. Phys Occup Ther Pediatr. 2006;26(3):19–38.

    PubMed  Google Scholar 

  74. Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, Pill R, Russell I, Stott N, Wilkinson C. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.

    Article  CAS  PubMed  Google Scholar 

  75. Royle PL, Bain L, Waugh NR. Sources of evidence for systematic reviews of interventions in diabetes. Diabet Med. 2005;22(10):1386–93.

    Article  CAS  PubMed  Google Scholar 

  76. Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St John PD, Viola R, Raina P. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.

    Article  PubMed  Google Scholar 

  77. Woodman J, Harden A, Thomas J, Brunton J, Kavanagh J, Stansfield C. Searching for systematic reviews of the effects of social and environmental interventions: a case study of children and obesity. J Med Libr Assoc. 2010;98(2):140–6.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Wright JM, Cottrell DJ, Mir G. Searching for religion and mental health studies required health, social science, and grey literature databases. J Clin Epidemiol. 2014;67(7):800–10.

    Article  PubMed  Google Scholar 

  79. Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.

    Article  PubMed  Google Scholar 

  80. Moseley AM, Sherrington C, Elkins MR, Herbert RD, Maher CG. Indexing of randomised controlled trials of physiotherapy interventions: a comparison of AMED, CENTRAL, CINAHL, EMBASE, Hooked on Evidence, PEDro, PsycINFO and PubMed. Physiotherapy. 2009;95(3):151–6.

    Article  PubMed  Google Scholar 

  81. Ramos-Remus C, Suarez-Almazor M, Dorgan M, Gomez-Vargas A, Russell AS. Performance of online biomedical databases in rheumatology. J Rheumatol. 1994;21(10):1912–21.

    CAS  PubMed  Google Scholar 

  82. Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.

    Article  PubMed  Google Scholar 

  83. Schoones JW, Vissers T. Re: “using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions”. J Clin Epidemiol. 2016.

  84. Bramer WM, Giustini D, Kramer BM. Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Syst Rev. 2016;5:39.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We wish to acknowledge and thank Videncentret, the Medical Library, Odense University Hospital and University Library of Southern Denmark for great and important help preparing this work. This work has been prepared as part of the SEARCH research group (Synthesis of Evidence And ResearCH). SEARCH is a Danish research group aiming at increase use of systematic review when prioritizing research, interpreting research results, and improving clinical practice. SEARCH also contributes to the development of methods for preparing systematic reviews, and evaluating the use of systematic reviews in research practice (meta-research).

Funding

The author(s) declare that they have received no funding.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

TAA, CJ and HL generated the concept and design of the study. TAA screened reviews for eligibility, undertook the data collection and analysis. All authors participated in the development of the manuscript from its early stages. All authors contributed to, read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Author information

Authors and Affiliations

  1. Department of Physiotherapy, Holbaek University Hospital, Holbaek, Denmark

    Thomas Aagaard

  2. Research Unit for Musculoskeletal Function and Physiotherapy, Institute for Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense, Denmark

    Thomas Aagaard, Hans Lund & Carsten Juhl

  3. Center for Evidence-based practice, Bergen University College, Bergen, Norway

    Hans Lund

  4. Department of Rehabilitation, Copenhagen University Hospital, Herlev, Gentofte, Denmark

    Carsten Juhl

Authors
  1. Thomas Aagaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hans Lund
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carsten Juhl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Aagaard.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aagaard, T., Lund, H. & Juhl, C. Optimizing literature search in systematic reviews – are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders?. BMC Med Res Methodol 16, 161 (2016). https://0-doi-org.brum.beds.ac.uk/10.1186/s12874-016-0264-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12874-016-0264-6

Keywords

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us