Skip to main content

Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)?

Abstract

Randomised trials are a central component of all evidence-informed health care systems and the evidence coming from them helps to support health care users, health professionals and others to make more informed decisions about treatment. The evidence available to trialists to support decisions on design, conduct and reporting of randomised trials is, however, sparse. Trial Forge is an initiative that aims to increase the evidence base for trial decision-making and in doing so, to improve trial efficiency.

One way to fill gaps in evidence is to run Studies Within A Trial, or SWATs. This guidance document provides a brief definition of SWATs, an explanation of why they are important and some practical ‘top tips’ that come from existing experience of doing SWATs. We hope the guidance will be useful to trialists, methodologists, funders, approvals agencies and others in making clear what a SWAT is, as well as what is involved in doing one.

Peer Review reports

Introduction

Randomised trials are a central component of all evidence-informed health care systems and they form a body of evidence that can help health care users, health professionals, policy-makers and others make informed choices about the effectiveness of treatments and therapies that they use and provide. The same is not true for trial design, conduct and reporting decisions, which are generally uninformed by evidence because there is little relevant evidence to turn to.

Trial Forge (www.trialforge.org) [1] is an initiative that aims to increase the evidence base for trial decision-making and, in doing so, to improve trial efficiency. One way to fill gaps in evidence is to run Studies Within A Trial, or SWATs. Descriptions of SWATs have been published [2, 3] but here we provide some guidance that provides a brief definition of a SWAT, an explanation of why they are important and some practical ‘top tips’ that come from existing experience of doing SWATs. We hope the guidance will be useful to trialists, methodologists, funders, approvals agencies and others in making clear what a SWAT is, as well as what is involved in doing one. We encourage them to use the text freely on their own websites and materials, with appropriate acknowledgement.

The text is based on discussions held during and after a 1-day meeting in Aberdeen, UK on 23 March 2017 as part of the Trial Forge initiative. This paper is the first Trial Forge Guidance document and there will be more guidance documents in the future, each providing what we hope is clear help and guidance around an issue relevant to improving the evidence base for trial decision-making. Trial methodologists and other stakeholders will be consulted to determine the topic areas and scope for future guidance.

What is a Study Within A Trial (SWAT)?

A SWAT is a self-contained research study that has been embedded within a host trial with the aim of evaluating or exploring alternative ways of delivering or organising a particular trial process.

Why do we need to do SWATs?

The need for randomised trials to evaluate the effects of health care interventions, such as new drugs and other treatments, is a familiar concept to people working in health and health research. The result of the trial provides evidence on how effective (or ineffective) the intervention is, helping both practitioners and health care users to make well-informed decisions about using it. These trials are central to improvements in health and social care.

Therefore, it is essential that the trials themselves are done in the most effective ways and one way to do this is to use the same types of evaluation to investigate and improve the processes of how we do randomised trials.

Unfortunately, only a small number of such studies have been done and there is very little evidence to allow researchers to make well-informed decisions about how to do their trials [1]. This means that researchers doing trials, funders paying for them and patients taking part in them cannot always be sure that the way the trial is being done is as effective and efficient as it could be. The most obvious example of this is that the evidence available to support trial teams to recruit patients to their trials is very thin, despite recruitment being a recognised problem for many trials [4, 5] and being identified as the top priority for research into trial methods [6].

One way of increasing this evidence base is to do a Study Within A Trial (SWAT) [2, 3]. SWATs evaluate alternative ways of doing a trial process (e.g. recruiting patients, helping them to stay in the study, or reporting the findings) to provide evidence about how to improve the process.

Key features of a typical SWAT

  • It seeks to resolve important uncertainties about the processes used in trials

  • It is embedded within a host trial

  • It must not affect the scientific integrity of the host trial, its rationale or outcome measures

  • It should have a formal protocol, just like the host trial

  • It can be evaluated in a single trial but is well-suited for running across more than one host trial, either at the same time or sequentially

  • It will provide data to inform the design and conduct of future trials but might also provide data to inform decisions about the ongoing host trial

For some practical considerations regarding SWATs, see Table 1. The information in the table comes mainly from experience with SWATs in the UK and Ireland but is likely to be useful for SWATs planned in other countries too.

Table 1 Practical things to consider when planning a Study Within A Trial (SWAT)

An example of a SWAT

Most trials have a Participant Information Leaflet (PIL), which tells a potential participant about the trial. The trial team uses this to offer information to potential trial participants in a way that it hopes will also help recruitment (and perhaps retention) whilet adhering to ethical standards.

The Systematic Techniques for Assisting Recruitment to Trials (START) programme (http://research.bmh.manchester.ac.uk/mrcstart/) developed a SWAT to evaluate the effect of a bespoke, tailored and user-tested PIL on recruitment compared with a standard PIL. The bespoke method of developing a PIL is expensive so it is important to know how much, if any, difference it makes to the trial. For instance, if the aim is to increase recruitment, it is essential to know that recruitment is indeed increased with the bespoke PIL compared with a standard PIL before using it in a future trial. The SWAT has already been evaluated in several trials (see, for example [7, 8]) and the emerging results are shown in Fig. 1. This meta-analysis shows that the current estimate for the effect on recruitment is small and not statistically significant: 1% improvement (95% confidence interval, − 1–2%).

Fig. 1
figure 1

Meta-analysis of three evaluations of the effect on trial recruitment of a bespoke, tailored and user-tested method of developing a Participant Information Leaflet

In other words, the bespoke PIL had little or no effect on recruitment compared with a standard PIL. By approaching investigators, encouraging them to embed an evaluation of the two types of PIL into their trials and then coordinating the analysis of data from those trials that did, the START programme’s coordinated, collaborative approach of embedding a SWAT evaluation in trials involving over 6600 people now provides an evidence base for researchers trying to decide on whether to develop a bespoke PIL for their trial.

Other examples of questions that could be addressed in SWATs include:

  • Comparing the effect of different financial incentives to encourage patients to complete a questionnaire used to collect trial outcomes

  • Determining whether recruitment is boosted if non-responders to postal invitations to join a trial are reminded by telephone

  • Evaluating the effect on recruitment and retention of a two-stage Participant Information Leaflet (i.e. the leaflet is delivered to participants in two parts: a short ‘key points’ version together with a longer version containing more detail) compared with a standard, single-stage leaflet

  • Evaluating the effect on data quality of providing site staff with face-to-face data entry training compared with Skype or video-conference training

  • Exploring which type of information participants think would best recognise the value of their contribution to the host trial results

There are plenty of uncertainties around how we should do trials, so it is highly likely that a trial team can find something that is interesting to them and worth investigating in a SWAT. For example, the Prioritising Recruitment in Randomised Trials (PRioRiTy) project (http://priorityresearch.ie/) generated a list of priority areas for recruitment research and many of these could be addressed by SWATs.

What happens to SWAT results?

Just as health researchers have a responsibility to make the findings of their clinical trials available, the findings from SWATs should be made public, so that the evidence base available for future decisions can increase. The findings can then be picked up by systematic reviewers and others who synthesise research evidence. The person doing the SWAT can facilitate this by, for example, contacting those who have done a relevant Cochrane Methodology Review (who will be updating it), to let them know about the SWAT. This means that even if the SWAT is not published separately itself, its results can be incorporated into the review.

SWAT results can also directly inform decisions within the host trial where uncertainty exists as to the best method to use for a particular process. The BEEP trial (https://www.journalslibrary.nihr.ac.uk/programmes/hta/126712/#/) is using a SWAT [9] and an interim analysis to help make a decision about the retention strategies to be used in the trial. A web-based trial linked to antibiotic prescribing also used a SWAT to make a decision about the best way to invite participants to take part in the second stage of the trial [10]. Although both of these SWATs provide useful information for other trials, they were designed to directly inform process decisions taken within the host trial.

The SWAT repository

Queen’s University Belfast in Northern Ireland hosts a SWAT repository (go.qub.ac.uk/SWAT-SWAR), which contains a list of prepared SWAT outlines. A form to register a new SWAT is also available online at http://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/ApplicationForms/SWATApplication/. Registering SWATs on the repository helps to avoid unnecessary duplication of effort and provides other researchers with ideas for how they might test the processes they will use in their own clinical trial.

The PRioRiTy repository

If your SWAT addresses one of the top 20 PRioRiTy research questions it can also be added to the PRioRiTy online repository (http://priorityresearch.ie/) which is a collection of ongoing research specific to recruitment to trials. This repository is hosted by the Health Research Board – Trials Methodology Research Network in Ireland. Ideally, these methodology studies should be included in both the PRioRiTy and the SWAT repository to help people to find them.

Abbreviations

PIL:

Participant Information Leaflet

PRioRiTy:

Prioritising Recruitment in Randomised Trials

START:

Systematic Techniques for Assisting Recruitment to Trials

SWAT:

Study Within A Trial

References

  1. Treweek S, Altman DG, Bower P, Campbell M, Chalmers I, Cotton S, et al. Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform. Trials. 2015;16:261.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Anon. Education section—Studies Within A Trial (SWAT). J Evid Based Med. 2012;5:44–5.

    Article  Google Scholar 

  3. Clarke M, Savage G, Maguire L, McAneney H. The SWAT (study within a trial) programme; embedding trials to improve the methodological design and conduct of future research. Trials. 2015;16(Suppl 2):209.

    Article  Google Scholar 

  4. Walters SJ, Bonacho dos Anjos Henriques-Cadby I, Bortolami O, Flight L, Hind D, Jacques RM, et al. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ Open. 2017;7:e015276.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Sully BGO, Julious SA, Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14:166.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Smith CT, Hickey H, Clarke M, Blazeby J, Williamson P. The trials methodological research agenda: results from a priority setting exercise. Trials. 2014;15:32.

    Article  Google Scholar 

  7. Cockayne S, Fairhurst C, Adamson J, et al. An optimised patient information sheet did not significantly increase recruitment or retention in a falls prevention study: an embedded randomised recruitment trial. Trials. 2017;18:144.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Man MS, on behalf of the Healthlines Study Group, Rick J, Bower P, on behalf of the MRC-START Group. Improving recruitment to a study of telehealth management for long-term conditions in primary care: two embedded, randomised controlled trials of optimised patient information materials. Trials. 2015;16:309.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Montgomery A, Williams H, Bradshaw L, Chalmers J. SWAT 25: two-by-two factorial randomised trial to evaluate strategies to improve follow-up in a randomised prevention trial. http://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/FileStore/Filetoupload,545015,en.pdf. Accessed on the SWAT repository on 18 Sep 2017.

  10. Treweek S, Barnett K, MacLennan G, Bonetti D, Eccles M, Francis J, Jones C, Pitts NB, Ricketts IW, Weal M, Sullivan F. E-mail invitations to general practitioners were as effective as postal invitations and were more efficient. J Clin Epidemiol. 2012;65:793–7.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Rosemary Humphreys for her contributions to the discussions in Aberdeen on 23 March 2017. The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health Directorates.

Funding

The work described here was funded in part by the UK Network of MRC Hubs for Trials Methodology Research and the Irish Trials Methodology Research Network.

Availability of data and materials

Not applicable

Author information

Authors and Affiliations

Authors

Contributions

ST and DD had the original idea for this guidance. MClarke, SC, DD, AF, ST, HG, KG and MW contributed to planning the meeting on 23 March 2017. PB, SB, CC, JC, MCampbell, SC, AF, EF, HG, SG, JJ, RL, AP, CR, LR, FS, DT, LT, ST, MW and PW attended the meeting and contributed to discussions. All authors contributed to writing this paper and approved the final draft.

Corresponding author

Correspondence to Shaun Treweek.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

MC, SC, DD, AF, ST, HG, KG and MW are members of the Trial Forge Steering Group. MC is responsible for the SWAT repository. EF is employed by BioMed Central and ST is a Senior Editor for Trials. All other authors report no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Treweek, S., Bevan, S., Bower, P. et al. Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)?. Trials 19, 139 (2018). https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-018-2535-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-018-2535-5