Skip to main content

Seeking adverse effects in systematic reviews of orthodontic interventions: protocol for a cross-sectional study

Abstract

Background

Before implementing healthcare interventions, clinicians need to weigh the beneficial and adverse effects of interventions. However, a large body of evidence has demonstrated that seeking and reporting of adverse effects is suboptimal in clinical trials and in systematic reviews of interventions. This cross-sectional study will investigate the status of this problem in orthodontics. This study will assess whether adverse effects were sought and whether findings related to adverse effects were reported in systematic reviews of orthodontic interventions in the five leading orthodontic journals and in the Cochrane Database of Systematic Reviews.

Methods

Systematic reviews of clinical orthodontic interventions published between 01 August 2009 and 31 July 2019 in the five leading orthodontic journals and in the Cochrane Database will be included. Empty reviews will be excluded. The reporting of outcomes on adverse effects will not determine eligibility, i.e., reviews will not be excluded, because they did not report usable data. Study selection and data extraction will be conducted independently by two authors. Our primary outcome will be the prevalence of systematic reviews of orthodontic interventions that sought any findings related to adverse effects in the included studies. Additional prevalence statistics will be calculated on a series of items related to seeking of adverse effects in the eligible reviews. All statistics will be calculated for (1) all journals together, (2) the group of five orthodontic journals and the Cochrane Database of Systematic Reviews separately, and (3) each individual journal separately. Chi-square tests of independence will be used to compare these groups.

Discussion

This study will assess whether adverse effects were sought in systematic reviews of orthodontic interventions. This knowledge is important, because reviews that present an incomplete picture on adverse effects can have unfavorable consequences for the end-users. Also not reporting that no adverse effects were assessed in eligible studies included in a systematic review can mislead pertinent stakeholders. Our findings could have policy implications for making judgments on accepting or rejecting an intervention systematic review for publication, for example, by directing editors and peer-reviewers to adopt the various items on adverse effects defined in the MECIR standards and in the PRISMA harm checklist.

Peer Review reports

Background

Making balanced decisions on healthcare interventions requires reliable evidence on both their beneficial and adverse effects. In the Cochrane systematic reviews of interventions, it is therefore mandatory to seek both types of outcomes and include at least one undesirable outcome as a primary outcome measure [1, 2]. In both Cochrane and non-Cochrane reviews of orthodontic interventions, we will assess whether adverse effects were sought and whether findings related to adverse effects were reported.

Since its foundation in 1993, Cochrane has set the standard for medical research-synthesis publications [3]. Systematic reviews with or without meta-analyses are the core of such syntheses and are the foundations for evidence-based practice guidelines and policy. Cochrane reviews of interventions aim at including outcomes that are likely to be important for patients, clinicians, the general public, guideline developers, administrators, and policy makers [2]. Cochrane states: “It is critical that outcomes used to assess adverse effects as well as outcomes used to assess beneficial effects are among those addressed by a review” (chapter 5.4.1) [2]. This issue is important, because a balanced perspective of an intervention can only be obtained when both types of outcomes are assessed and reported with the same rigor. Cochrane has formulated the following definition of an adverse effect: “An adverse event for which the causal relation between the intervention and the event is at least a reasonable possibility” [4, 5]. We adopted Cochrane’s definitions of adverse effects, systematic reviews, and interventions reviews in this manuscript (Table 1) [4,5,6].

Table 1 Glossary of terms

Numerous epidemiological studies have shown that adverse effects of interventions are often under-assessed or under-reported in primary research studies [7,8,9,10,11]. In addition, much information on adverse events remains unpublished and the number and range of these events are higher in unpublished compared to published versions of the same study [12]. To improve the reporting of harms in randomized trials, an extension of Consolidated Standards of Reporting Trials (CONSORT) Statement was developed [13]. The reporting of adverse events has improved over time since the publication of this extension, but was still suboptimal for a wide variety of clinical trials [9, 11, 14]. Systematic reviewers have an important role in bringing these issues to the foreground. However, epidemiological studies have shown that seeking and reporting adverse effects of interventions is also suboptimal in systematic reviews [15,16,17,18]. In 2016, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) harm checklist [19] to improve harm reporting in systematic reviews was published, but the consequences of this checklist are still unknown.

In this study, we will assess whether adverse effects were sought and reported in systematic reviews of orthodontic interventions. We will scrutinize such reviews in the five leading orthodontic journals and those registered in the Cochrane Database of Systematic Reviews. Reporting on pain as a result of tooth movement and the various categories of known orthodontic adverse effects as defined by Preoteasa et al. [20] will be assessed in these reviews (Table 2). Scoping searches in the orthodontic literature confirmed the knowledge gaps on our research questions. Our pilot studies on intervention reviews of the Cochrane Database of Systematic Reviews and those published in the five leading orthodontic journals quantified these gaps and further showed the need to undertake this research study. Addressing our research objectives is crucial for patients, clinicians, researchers, policy makers, and research sponsors. These questions are particularly important, because systematic reviews are increasingly consulted by patients [21].

Table 2 Adverse effects hypothetically linked to orthodontic interventions [20]

Objectives

The main research question of this cross-sectional study is the following: “Do reviewers seek adverse effects in systematic reviews of orthodontic interventions?” To address this question, we have defined the following objectives:

  • To calculate the prevalence of eligible systematic reviews of orthodontic interventions that defined seeking of adverse effects as a research objective of the review

  • To calculate the prevalence of eligible systematic reviews of orthodontic interventions that sought any findings related to adverse effects in the included studies

  • To calculate the prevalence of eligible systematic reviews of orthodontic interventions that considered and discussed (weighed) potential adverse effects of the intervention anywhere in the review

  • To calculate the prevalence of each type of adverse effect sought in the review

Methods

We used the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015 statement as the guideline for reporting this protocol [22, 23]. The PRISMA-P checklist is included as Additional file 1. Figure 1 represents the flow diagram of our research methods. Our first step was to conduct scoping searches to identify knowledge gaps and prioritize research questions on seeking and reporting of adverse effects in systematic reviews of orthodontic interventions. Two reviewers (PS and RMR) subsequently conducted pilot tests to assess the validity of these questions and the research methods and to fine-tune them. The sample size for the pilot test was calculated a priori [24], and random numbers were generated to select pilot systematic reviews [25]. The procedures for our pilot tests are reported in Additional file 2. In the following sections, we presented our planned methods based on these pilot tests.

Fig. 1
figure 1

Flow diagram of the research methods

Eligibility criteria

Study designs

  • We will include systematic reviews of orthodontic interventions. The definition of a systematic review, an intervention review, and orthodontic interventions listed in the Glossary of terms will be used to assess whether a review is eligible (Table 1).

  • We will exclude (1) non-interventional reviews such as “Methodology,” “Diagnostic,” “Qualitative,” and “Prognostic”; (2) rapid and scoping reviews; (3) systematic reviews that focus exclusively on adverse effects of interventions; and (4) systematic reviews of interventions that did not find any eligible studies (empty reviews).

Participants

  • We will include systematic reviews on any type of patients undergoing orthodontic interventions, i.e., patients of any health status, sex, age, demographics, and socio-economic status.

  • We will exclude (1) intervention reviews that focus exclusively on patients with congenital anomalies, for example, with cleft lip and palate, and (2) systematic reviews of animal or laboratory studies.

Interventions

  • We will include the following: (1) Systematic reviews that assess the effects of clinical orthodontic interventions. Clinical orthodontic interventions refer to the use of any type of orthodontic appliances that are used to move teeth or change the jaw size or position for orthodontic purposes. (2) Systematic reviews of interventions with appliances to maintain or stabilize the results of orthodontic treatment, for example, retainers. (3) Systematic reviews of orthodontic interventions that compare the effects of orthodontic treatment with or without additional interventions such as pharmacological or small surgical procedures, e.g., periodontal or implant surgery.

  • We will exclude (1) systematic reviews in which patients receive orthodontic treatment, but in which the effects of other interventions, e.g., periodontal surgery, are compared and not the effects of orthodontic interventions; (2) systematic reviews of interventions in which orthodontic appliances are specifically used for other purposes, e.g., changing jaw positions to treat respiration or temporomandibular disorders; and (3) systematic reviews of orthodontic interventions that included orthognathic surgery.

  • No exclusion criteria will be applied to the characteristics of the operator who conducted the interventions.

Outcomes

  • Any adverse effect of an orthodontic intervention scored at any endpoint or timing will be eligible.

  • The effects of orthodontic interventions do not refer just to outcomes related to tooth and jaw size and positions, but also to broader outcomes such as periodontal health, esthetic changes, the health of the temporomandibular joint, patient health experiences, and economic issues associated with the intervention.

  • The reporting of outcomes on adverse effects will not determine the eligibility of reviews for this cross-sectional study, i.e., reviews will not be excluded because they did not provide “usable” data [2].

Setting

  • No exclusion criteria will be applied to the type of setting, e.g., university or private practice, etc., in which the interventions were conducted.

Information sources

We will manually search eligible systematic reviews between 01 August 2009 and 31 July 2019 in the Cochrane Database of Systematic Reviews [26] and in the websites of the five leading orthodontic journals. We consulted the journal citation reports by Clarivate Analytics [27] to identify the five leading orthodontic journals based on their impact factor. Based on these reports, the following orthodontic journals were included: European Journal of Orthodontics [EJO], American Journal of Orthodontics and Dentofacial Orthopedics [AJODO], Angle Orthodontist, The Korean Journal of Orthodontics, and Orthodontics and Craniofacial Research. Recently launched orthodontic journals, i.e., covering less than 10 years of journal publication, will not be eligible. The first of August 2009 was chosen as the incept data for our searches, because it coincides with the launch of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement and guidance on 21 July 2009 [28, 29].

Study records

Data management

  • All study selection and data extraction procedures will be conducted by two authors (PS and RMR) independently.

  • Our pilot tests were also used to train both reviewers in applying our methods consistently and to calibrate them [23].

  • Disagreement on the eligibility of a paper or the extraction of data will be resolved through (1) discussions between reviewers, (2) rereading the pertinent paper, or (3) contacting its authors by email [28]. Persistent disagreements will be resolved through the consultation of a methodologist (SB).

  • All eligible systematic reviews will be downloaded as PDFs, and all data will be extracted to an Excel spreadsheet [30].

Selection process

  • All titles and abstracts will be screened for eligibility in the websites of the five orthodontic journals. We will search the section “Dentistry and Oral health” for eligible reviews in the Cochrane Database of Systematic Reviews [26].

  • When updates of reviews are identified, we will only consider the latest version.

  • Authors suspect of multiple publications of the same systematic review will be contacted by email. We plan to consider the first publication, but this decision will be weighed on a case-by-case basis. Our rationale for these decisions will be reported in the completed study.

  • A PRISMA flow diagram will illustrate our selection procedures [28, 29].

  • All eligible and excluded systematic reviews will be presented in tables. The rationale for exclusion will be listed for each excluded review.

Data collection process

  • Eligible studies and their pertinent supplemental files will be merged into binder PDFs, and multiple search terms will be applied to facilitate data extraction [31, 32].

  • We consulted various articles on adverse effects [4, 13, 18, 19, 33, 34] and thesauri to develop these search terms. A table with all search terms is listed in Additional file 3.

  • All pertinent data items will be extracted using our pilot tested data collection forms. These forms are presented in Additional file 4 and incorporate all our research questions. We consulted the PRISMA [28, 29] and the PRISMA-P [22, 23] checklists to develop these data collection forms.

  • Criteria for scoring the pertinent data items are defined in these forms.

  • We will search the entire eligible review, i.e., the text, tables, figures, and supplemental files. The plain language summary in eligible Cochrane systematic reviews will not be scrutinized for data items.

  • Modifications made in the collection forms during data extraction will be reported in the section “Differences between the protocol and review” together with the rationale for these changes.

Scoring adverse effects of orthodontic interventions

  • We will adopt a priori the various categories of known orthodontic adverse effects as defined by Preoteasa et al. [20], which were divided into two main types: local and systemic, with their pertinent subtypes (Table 2).

  • We will also consider pain as a result of tooth movement and additional adverse effects of orthodontic interventions that are identified post hoc, i.e., during data extraction, and are not listed in Table 2. We will explain the rationale for including specific additional effects as adverse and will produce a framework for categorizing them.

  • Ambiguous outcomes that could be interpreted as either beneficial or adverse will not be scored as “adverse.” We will also present the rationale for this score. Ambiguous outcomes will only be scored as adverse when the authors of the pertinent review define these outcomes as such.

Outcomes and statistical analyses

  • All research questions are presented in flow diagrams (Fig. 2).

  • All planned outcomes are presented in a summary of findings table (Table 3).

  • All prevalence data will be calculated and reported with their 95% confidence levels.

  • Prevalence statistics will be calculated for (1) all journals together, (2) the group of five orthodontic journals and the Cochrane Database of Systematic Reviews separately, and (3) each individual journal separately. Comparisons between these statistics will be calculated. These statistics will be compared with chi-square tests of independence. We will report the value of chi-square, the degrees of freedom (df), and the p value. A p value of < 0.05 will be considered to be statistically significant. We will use Stata software (Stata Corporation, College Station, TX, USA) version 15 for all the statistical analyses [35].

  • We will report all outcomes that will be introduced or eliminated post hoc together with the rationale for inclusion or exclusion.

Fig. 2
figure 2

Flow diagram “seeking adverse effects of interventions in systematic reviews of orthodontic interventions”

Table 3 Summary of findings

Reporting of the research study and data management

  • We will adopt The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement as the guideline for reporting the completed cross-sectional study [36].

  • We prepared a data management plan for the long-term storage of our research data [37]. This plan guarantees that (1) all our project data will be made freely available and (2) our submitted article will be accompanied by additional files with all raw data of the completed study or with a link to a repository where these files will be deposited. In the latter case, we will register our repository in the Registry of Research Data Repositories [38]. (3) Our project data will be presented in a format that permits other scientists to understand, cite, and reuse the data. (4) Sensitive data will be protected. (5) Our data management plan will be frequently reassessed and updated if necessary [37, 38].

Differences between the protocol and the completed study

All differences between the protocol and the final research study will be reported together with the rationale for these changes. We will also present the consequences of these modifications on the magnitude, direction, and validity of the outcomes [39].

Discussion

Strengths

We point at four key strengths of this research study. First, extensive scoping searches and pilot studies were conducted to fine-tune our research questions and procedures. Our pilot studies also confirmed the importance of our research questions. Second, the research team consisted of two topic experts (PS and RMR) and two methodologists (RMR and SB). Third, all study selection and data extraction procedures were conducted by two operators (PS and RMR) independently. Fourth, this study will permit reproducibility, because we will publish the protocol a priori and all raw data of the completed study will be reported in additional files or will be deposited in an open access repository [37, 40].

Limitations

The limitations of this research study include the following: (1) It does not cover all journals that have published orthodontic intervention systematic reviews, but only a subgroup, i.e., those published in the five leading orthodontic journals and in the Cochrane Database of Systematic Reviews. However, we expect that the choice of this subgroup of the leading orthodontic literature will produce outcomes that will underestimate the true severity of the problem. (2) Including only systematic reviews of orthodontic interventions published in the last 10 years could introduce the risk of publication bias. However, we chose this period, because it will represent the actual knowledge status on assessing adverse effects in orthodontic intervention systematic reviews. Further, this period coincides with the launch in 2009 of the PRISMA reporting checklist, which is an important update on how to report items in systematic reviews [28, 29].

Importance and beneficiaries

In this research study, we will assess whether adverse effects were sought and reported in both Cochrane and non-Cochrane systematic reviews of orthodontic interventions. This is important, because of the following: (1) The validity of the findings of systematic reviews of interventions depends on a balanced presentation of both the benefits and adverse effects of the intervention [19]. (2) There is a large body of evidence that has demonstrated that seeking and reporting of adverse effects is suboptimal in a wide variety of clinical trials [7,8,9,10,11]. Systematic reviewers can have a crucial role as whistle blowers by bringing these knowledge gaps to the foreground. However, their position can also be damaging, because reviews that present an incomplete picture on these gaps can have unfavorable consequences for the end-users. For example, not reporting that no adverse effects were assessed in eligible studies included in a systematic review can mislead readers.

Our findings could have policy implications for making judgments on accepting or rejecting a systematic reviews of orthodontic interventions for publication, for example, by directing editors and peer-reviewers to adopt the various items on adverse effects defined in the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards [1] and the PRISMA harm checklist [19]. Patients, clinicians, researchers, editors, peer-reviewers, guideline developers, policy makers, and research funders will all benefit from the findings of this research study.

Abbreviations

EQUATOR Network:

Enhancing the Quality and Transparency Of health Research Network

MECIR:

Methodological Expectations of Cochrane Intervention Reviews

PRISMA:

Preferred Reporting Items for Systematic reviews and Meta-Analyses

PRISMA-P:

Preferred Reporting Items for Systematic review and Meta-Analysis-Protocols

References

  1. Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews (MECIR). Standards for the conduct and reporting of new Cochrane Intervention Reviews, reporting of protocols and the planning, conduct and reporting of updates. London: Cochrane; 2016. [online] Available from: https://community.cochrane.org/mecir-manual (Accessed 7 Dec 2018)

    Google Scholar 

  2. O’Connor D, Green S, Higgins JPT. Chapter 5: defining the review question and developing criteria for including studies. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011): The Cochrane Collaboration; 2011. [online] Available from: http://handbook-5-1.cochrane.org/. (Accessed 20 Aug 2018).

  3. Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018;555(7695):175–82.

    Article  CAS  Google Scholar 

  4. Loke YK, Price D, Herxheimer A. Chapter 14: adverse efffects. In: JPT H, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011): The Cochrane Collaboration; 2011. [online] Available from: http://handbook-5-1.cochrane.org/. (Accessed 20 Aug 2018).

  5. Glossary of terms in the Cochrane Collaboration. Version 4.2.5. Updated May 2005. [online] Available from: http://aaz.hr/resources/pages/57/7.%20Cochrane%20glossary.pdf (Accessed 20 Aug 2018).

  6. Cochrane library. [online] Available from: https://www.cochranelibrary.com/about/about-cochrane-reviews (Accessed 20 Aug 2018).

  7. Bagul NB, Kirkham JJ. The reporting of harms in randomized controlled trials of hypertension using the CONSORT criteria for harm reporting. Clin Exp Hypertens. 2012;34(8):548–54 doi: 10.3109/10641963.2012.681724. Epub 2012 May 9.

    Article  Google Scholar 

  8. Hodkinson A, Kirkham JJ, Tudur-Smith C, Gamble C. Reporting of harms data in RCTs: a systematic review of empirical assessments against the CONSORT harms extension. BMJ Open. 2013;3(9):e003436. https://0-doi-org.brum.beds.ac.uk/10.1136/bmjopen-2013-003436.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Péron J, Maillet D, Gan HK, Chen EX, You B. Adherence to CONSORT adverse event reporting guidelines in randomized clinical trials evaluating systemic cancer therapy: a systematic review. J Clin Oncol. 2013;31(31):3957–63. https://0-doi-org.brum.beds.ac.uk/10.1200/JCO.2013.49.3981 Epub 2013 Sep 23.

    Article  PubMed  Google Scholar 

  10. Pitrou I, Boutron I, Ahmad N, Ravaud P. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med. 2009;169(19):1756–61.

    Article  Google Scholar 

  11. Smith SM, Chang RD, Pereira A, Shah N, Gilron I, Katz NP, Lin AH, MP MD, Rappaport BA, Rowbotham MC, Sampaio C, Turk DC, Dworkin RH. Adherence to CONSORT harms-reporting recommendations in publications of recent analgesic clinical trials: an ACTTION systematic review. Pain. 2012;153(12):2415–21. https://0-doi-org.brum.beds.ac.uk/10.1016/j.pain.2012.08.009 Epub 2012 Sep 15.

    Article  PubMed  Google Scholar 

  12. Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Med. 2016;13(9):e1002127. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pmed.1002127 eCollection 2016 Sep.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Ioannidis JP, Evans SJ, Gøtzsche PC, O’Neill RT, Altman DG, Schulz K, Moher D, CONSORT Group. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141(10):781–8.

    Article  Google Scholar 

  14. Haidich AB, Birtsou C, Dardavessis T, Tirodimos I, Arvanitidou M. The quality of safety reporting in trials is still suboptimal: survey of major general medical journals. J Clin Epidemiol. 2011;64(2):124–35. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2010.03.005 Epub 2010 Jun 17.

    Article  PubMed  Google Scholar 

  15. Hopewell S, Wolfenden L, Clarke M. Reporting of adverse events in systematic reviews can be improved: survey results. J Clin Epidemiol. 2008;61(6):597–602. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2007.10.005 Epub 2008 Apr 14.

    Article  PubMed  Google Scholar 

  16. Hammad TA, Neyarapally GA, Pinheiro SP, Iyasu S, Rochester G, Dal Pan G. Reporting of meta-analyses of randomized controlled trials with a focus on drug safety: an empirical assessment. Clin Trials. 2013;10(3):389–97. https://0-doi-org.brum.beds.ac.uk/10.1177/1740774513479467. Epub 2013 Mar 18.

    Article  PubMed  Google Scholar 

  17. Mahady SE, Schlub T, Bero L, Moher D, Tovey D, George J, Craig JC. Side effects are incompletely reported among systematic reviews in gastroenterology. J Clin Epidemiol. 2015;68(2):144–53.

    Article  Google Scholar 

  18. Zorzela L, Golder S, Liu Y, Pilkington K, Hartling L, Joffe A, Loke Y, Vohra S. Quality of reporting in systematic reviews of adverse events: systematic review. BMJ. 2014;348:f7668. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.f7668 Review.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, Moher D, Vohra S, PRISMA Harms Group. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352:i157. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.i157.

    Article  PubMed  Google Scholar 

  20. Preoteasa CT, Ionescu E, Preoteasa E. Chapter 18: risks and complications associated with orthodontic treatment. In: Bourzgui F, editor. Orthodontics-basic aspects and clinical considerations; 2012. under CC BY 3.0 license. www.intechopen.com. [online] Available from: https://www.intechopen.com/books/orthodontics-basic-aspects-and-clinical-considerations/risks-and-complications-associated-with-orthodontic-treatment (Accessed 20 Aug 2018).

    Google Scholar 

  21. Reeves BC. Reporting of harms in systematic reviews and their primary studies. BMJ. 2014;349:g6819. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.g6819.

    Article  PubMed  Google Scholar 

  22. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1.

    Article  Google Scholar 

  23. Shamseer L, Moher D, Clarke M, Ghersi D, Deceased LA, Petticrew M, Shekelle P, Stewart LA, the PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.

    Article  Google Scholar 

  24. Viechtbauer W, Smits L, Kotz D, Budé L, Spigt M, Serroyen J, Crutzen R. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015;68(11):1375–9. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2015.04.014 Epub 2015 Jun 6.

    Article  PubMed  Google Scholar 

  25. Random.org. Randomness and Integrity Services Ltd. [online] Available from: https://www.random.org/ (Accessed 20 Aug 2018).

  26. Cochrane library. Cochrane database of systematic reviews. [online] Available from: https://www.cochranelibrary.com (Accessed 20 Aug 2018).

  27. Clarivate Analytics. [online] Available from: https://clarivate.com/(Accessed 20 Aug 2018).

  28. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100 Epub 2009 Jul 21.

    Article  Google Scholar 

  29. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

    Article  Google Scholar 

  30. Microsoft Excel. [online] Available from: https://products.office.com/it-IT/excel (Accessed 20 Aug 2018).

  31. Adobe Merge PDFs, combine files into one PDF. [online] Available from: http://www.wikihow.com/Merge-PDF-Files (Accessed 20 Aug 2018).

  32. Acrobat for legal professionals. Searching and marking multiple words. [online] Available from: http://blogs.adobe.com/acrolaw/2010/04/searching-and-marking-multiple-words-in-a-pdf/ (Accessed 20 Aug 2018).

  33. Golder S, Loke YK, Zorzela L. Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. J Clin Epidemiol. 2013;66(3):253–60. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2012.09.013.

    Article  PubMed  Google Scholar 

  34. Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014;349:g6501. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.g6501.

    Article  PubMed  PubMed Central  Google Scholar 

  35. StataCorp. Stata statistical software: release 15. College Station: StataCorp LLC; 2017.

    Google Scholar 

  36. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, Initiative STROBE. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–7.

    Article  Google Scholar 

  37. Schiermeier Q. For the record. Making project data freely available for open science. Nature. 2018;555:403–5.

    Article  CAS  Google Scholar 

  38. Registry of Research Data Repositories. [online] Available from: https://www.re3data.org/ (Accessed 20 Aug 2018).

  39. Higgins JPT, Deeks JJ, Altman DG (editors). Chapter 16: special topics in statistics. In: Higgins JPT, Green S (editors), Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. [online] Available from: http://handbook-5-1.cochrane.org/ (Accessed 20 Aug 2018).

  40. Goodman SN, Fanelli D, Ioannidis JP. What does research reproducibility mean? Sci Transl Med. 2016;8(341):341 ps12. https://0-doi-org.brum.beds.ac.uk/10.1126/scitranslmed.aaf5027.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable

Funding

All expenses for preparing this protocol and for conducting the subsequent research study will be paid evenly by each author.

Availability of data and materials

Not applicable

Author information

Authors and Affiliations

Authors

Contributions

PS and RMR conceived and designed the study protocol for this cross-sectional study. RMR is the guarantor. PS and RMR conducted the pilot testing of the study selection procedures and data extraction forms and fine-tuned the research protocol after the pilot testing. SB provided support on methodological and statistical issues and assisted in the overall fine-tuning of this protocol. All authors read and approved the final protocol.

Corresponding author

Correspondence to Reint A. Meursinge Reynders.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

Reint Meursinge Reynders and Shandra Bipat are both Associate Editors for Systematic Reviews. The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Checklist for the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. (DOCX 33 kb)

Additional file 2:

Pilot tests. (DOCX 21 kb)

Additional file 3:

Search terms and their derivatives. (DOCX 15 kb)

Additional file 4:

Data collection forms. (DOCX 16 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Steegmans, P.A.J., Bipat, S. & Meursinge Reynders, R.A. Seeking adverse effects in systematic reviews of orthodontic interventions: protocol for a cross-sectional study. Syst Rev 8, 89 (2019). https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-019-1000-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-019-1000-1

Keywords