Design and study population

We conducted a pragmatic analysis before and after the introduction of the SBL care bundle (the exposure of interest) using longitudinal data collected retrospectively in 19 NHS Trusts in England across 9 clinical networks. These trusts were purposively selected from a larger cohort of maternity units that had participated in the 2015 NHS Tracker Survey to provide a geographically distributed sample throughout England and a range of birth rates and levels of neonatal care and to reflect a range of levels of implementation at the time of selection. Units from London were not included as they were participating in a stepped-wedge cluster trial of implementation of the Growth Assessment Protocol [12]. These trusts were deemed early adopters as they were pilot sites for implementing SBL in 2015 ahead of the national launch in March 2016. For the purposes of this analysis April 2015 was defined as the start of implementation in these 19 Trusts and outcomes were assessed at the nominal dates of April 2013 ("before") and April 2017 ("after") respectively, providing a minimum of two years of data before and after implementation of the SBL care bundle.

Retrospective data collection for the period April 2013 to October 2017 was carried out between September 2017 and January 2018. Recruitment (for surveys) took place between August and December 2017, following approval by the Health Research Authority in June 2017 (Reference 17/WM/0197). The study was registered on www.clinicaltrials.gov (NCT03231007) in July 2017 before commencement of data collection, and the evaluation protocol was published before commencing data analysis [11]. Analysis was performed on anonymized data. The study was reported according to STROBE guidelines [13].

Variables and data sources

Definitions of data collection methods and outcomes are described in detail in the protocol [11]. In brief, outcome data was collected from routine electronic clinical data held in Trust data repositories where this was available, the data being provided or converted to monthly aggregate totals (e.g. number of births per month). Anonymised postcode data were used to derive the index of multiple deprivation (IMD) and a mean decile computed for each Trust; where this was not provided an estimate was derived as the mean IMD of a surrounding area based on population size and birth numbers.

This was supplemented by clinical audits (20 sequential women per time point and Trust) of growth and birth monitoring practice in unselected women (pre and post implementation), small for gestational age births (pre and post implementation) and women attending hospital with RFM (post-implementation only). Additionally, in the post intervention period women were asked to complete a survey postpartum prior to discharge (informed consent) which included questions on smoking behaviour and interventions received along with experiences of RFM (S1 File). All health professionals involved in delivering maternity care were invited to participate in an online or paper survey regarding their views and experiences of the SBL care bundle (S2 File) if they had been employed in their current Trust prior to the launch of the care bundle initiative in April 2015. Data on the degree of implementation of the SBL care bundle was self-reported by Trusts and implementation scores were derived by applying a score to a Likert scale whether each intervention in each element was implemented: a) all of the time (score 3), b) most of the time (score 2), c) half of the time (score 1), d) not much of the time (score 0), e) never (score 0), or f) not relevant (score 0). These were totalled to give an implementation score for each element. As there were different numbers of interventions within individual elements, the overall implementation score for each Trust was calculated as the mean of the implementation scores for the four elements, each expressed as a percentage of the maximum score.

Statistical analysis

Electronic data on stillbirths other clinical outcomes and smoking rates was provided as, or converted to, monthly counts of outcomes and appropriate denominators. Outcomes were fitted using quasi-binomial models to allow for the possibility of overdispersion, except for scan rates, which were fitted using quasi-Poisson models. Generalised linear models with logarithmic link functions were used to estimate a linear trend over time. Trust was included as a covariate thus time trends are estimated on a within-Trust basis. From this model we derived estimates of the outcome rates at dates 2 years either side of the nominal SBL start date of April 2015 and estimated the risk- (or rate) ratio between these two time points as a measure of the change over the implementation period. Models additionally containing a step change at the nominal or reported implementation dates were also considered, but proved to be uninformative as step-changes could not be detected and therefore are not presented.

In order to investigate the relationship with reported implementation status, generalised linear mixed models were fitted with Trust as a random intercept and assessment-date implementation level, Care Level (Tertiary v Secondary), IMD (mean decile) and month as fixed effects. A risk-ratio between no and full implementation is reported.

Audit data and women’s questionnaire data were available for one or two time points and the outcomes are presented as proportions of women audited with exact Binomial 95% CI. Where there were data from 2 time points (pre and post implementation), risk ratios between the two time points were derived using binomial regression models with logarithmic link functions. Data analyses were conducted using R (www.R-project.org) [14].

The sample sizes for the various datasets were largely determined pragmatically by the constraints of time and the need to have a reasonable pre and post launch period to assess trends, with an inevitably staggered true implementation. Based on a prevalence of 4.7 normally formed singleton stillbirths per 1,000 total births in 2014 [4], the potential annual number of stillbirths detected was estimated as 470 per 100,000 total births across all study sites. Conservatively, a two year pre versus one year post-comparison was estimated to have 80% power to detect a drop in the primary stillbirth rate from 4.7 to 3.9/1000 –a 17% reduction [11].

Economic analysis

An economic analysis was carried out using the data generated from the comparisons outlined above alongside additional data reported by the Trusts on the resources they used to implement the SBL care bundle and the level of implementation they reported. These data were also used to estimate what annual costs and stillbirths avoided for the whole of England would be in the scenario that all Trusts implemented all elements of SBL care bundle.

The implementation cost consists of two parts: the direct cost of putting in place each element and the cost of the secondary effects (mode of delivery, induction of labour, ultrasound scans). The direct cost of implementing each element was estimated but it is not possible to determine which element the secondary effects relate to so they have been calculated for the SBL care bundle overall. No additional funding was provided to Trusts to implement the SBL therefore the direct implementation costs should be interpreted as the ’value’ of the SBL rather than additional funding required.

Unit costs were derived from published sources [15,16], NHS partners [17], and through email communication with training/software providers. The fees for externally-provided training courses were included in the implementation cost. In-house training was assumed to form part of routine, ongoing continuing professional development (CPD), and so the cost of staff time to complete training was not included in the primary analysis, but was explored in a sensitivity analysis.

The total annual birth rate for England was identified from national published data [18]. The time-series-adjusted stillbirth rates from before and after the implementation date were applied to this number to estimate the difference in the number of stillbirths before and after implementation of the SBL care bundle i.e. the number of stillbirths potentially avoided.

It was necessary to make a number of assumptions in order to estimate the resources and costs associated with the care bundle. As such the costs and outcomes reported should be interpreted as ‘best estimates’. The impact of varying some of the assumptions on costs and outcomes were explored in a series of one-way sensitivity analyses.

Stillbirth rates

Stillbirth rates declined from 4.2 to 3.4 per 1,000 births over the 4 year period, which was statistically significant (P<0.001), RR of 0.80 (Table 2). Fig 1 shows a steady decline in the rate over time, and we were unable to demonstrate any step changes associated with implementation of the SBL care bundle. Term singleton and stillbirths associated with SGA showed similar decline (Table 2). Pre-term stillbirth showed a slightly smaller decreased from 2.3 per 1,000 births to 1.9 per 1,000 births (Relative Risk 0.82, p = 0.014). We were unable to demonstrate any association between stillbirth rates and the reported level of implementation of the SBL care bundle (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Average total stillbirth rate across participating sites.

Comparison between pre and post implementation of the Saving Babies Lives care bundle demonstrates a reduction between pre- and post-implementation periods. Republished from Saving Babies’ Lives Project Impact and Results Evaluation (SPiRE): A mixed methodology study under a CC BY license, with permission from the University of Manchester, original copyright 2018.

https://doi.org/10.1371/journal.pone.0250150.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Clinical and service outcomes.

https://doi.org/10.1371/journal.pone.0250150.t002

Changes in secondary outcomes

Table 2 summarises the clinical and service outcomes derived from Trust database systems. There were modest increases in the numbers of preterm births, induction of labour and emergency Caesarean sections (CS) over the evaluation period. The number of ultrasound scans per pregnancy increased markedly from 3.5/pregnancy to 4.3/pregnancy in the 8 Trusts providing complete data (S2 Fig). Other Trusts provided partial scan data which was consistent with these trends, but not sufficiently complete for formal analysis. An increase in the number of ultrasound scans performed was associated with a reduction in stillbirth rate (Adjusted Rate Ratio of -0.14 (SE 0.06) stillbirths per additional scan performed P = 0.026). Increases were also seen in elective CS and admissions to NICU, although it must be noted that the quality of data regarding NICU admission was poor. None of these outcomes showed a significant association with implementation of the SBL care bundle.

Process outcomes

The number of women smoking at delivery declined from 14.3% to 11.8% (P<0.001) over the evaluation period. However, this reflects a decline in the proportion of women smoking at booking rather than an increase in the numbers ceasing smoking during pregnancy (Table 3). 70.1% of women reported being offered a CO test and 99.1% of these women reported accepting the offer, however a smaller proportion of women who smoked were offered referral to smoking cessation services (60.1%) and only 31.0% attended this service. Trusts reporting full implementation of the element had higher proportion of women having a CO test at booking (RR 7.95, Table 3), but no difference in the proportion of women referred for smoking cessation services (RR 1.11, Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Process outcomes.

https://doi.org/10.1371/journal.pone.0250150.t003

Over the time period of the evaluation, documentation of screening for SGA fetuses improved, with increased documentation of symphysis-fundal height and estimated fetal weight on fetal growth charts. The documentation of these parameters was also significantly higher in Trusts with full implementation of this element (RR 5.07 (symphysis-fundal height), RR 38.24 (estimated fetal weight)). The documented increase in screening for SGA fetuses and in the frequency of ultrasound scan was associated with a 59% increase in the detection rate of SGA fetuses from 33.8% to 53.7% (Table 3). The leaflet regarding RFM was reported to be received by the majority of women (73.1%), and attendance with RFM occurred in 77.3% of episodes of RFM. A lower proportion of cases used a checklist to guide management (52.2%) and management varied from practice recommended in SBL care bundle with 74.4% of women with RFM having a fetal heart rate tracing and 29.4% of women receiving an ultrasound scan. A large proportion of women reporting RFM had an induction of labour (54.7%; Table 3). Women attending trusts with full implementation of the SBL care bundle were more likely to receive the information leaflet about RFM (RR 2.87, Table 3).

We were unable to obtain sufficient data directly from participating Trusts regarding training in interpretation of fetal heart rate traces over the duration of implementation of the SBL care bundle. Data from the staff questionnaires showed that 56.5% of staff (n = 1,064) reported they had received training in the preceding year with 16.8% of those who had been trained reporting they had completed assessment in fetal heart rate interpretation. Staff employed the “buddy system” where at least two professionals interpreted the fetal heart rate trace in 50.2% of audited cases. Trusts with full implementation were more likely to have used the CTG “buddy system” and a standardised recording of the fetal heart rate assessment (RR 33.00).

Staff views

1,064 health professionals completed the staff survey (details of respondents can be found in S3 File); 78% of respondents were midwives. Overall, 58% (584/905) of respondents were aware of the SBL care bundle, with lowest rates of awareness in junior doctors and ultrasonographers (47% (7/15) and 32% (11/34) of respondents respectively). Almost all staff perceived an increased demand for ultrasound scans (97%, Table 4), rates of induction of labour (98%) and to a lesser extent Caesarean sections (80%). A significant proportion of respondents (41%) perceived that the stillbirth rate had decreased in their unit, although the same proportion perceived it was unchanged (Table 4). Most staff 82% (807/983) agreed with the statement that their unit was actively doing things to improve the safety of mothers and babies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Staff views about changes in practice and outcomes in their Trust over the time frame of adoption of the SBL care bundle (n = 1,064).

https://doi.org/10.1371/journal.pone.0250150.t004

Economic analysis

There were 666,025 births (live births and stillbirths) in England in 2016 [18]. Based on data observed from the study sites, the number of women booked is on average 14% higher than the number of births, therefore, it was estimated that there were 759,299 bookings in the same period. The primary estimated cost of implementing the care bundle for the first year in England is £93,116,650, or approximately £140 per birth. The direct cost (excluding secondary costs) is £3,650,357 (4%) of the total cost. The full breakdown of costs are reported in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Estimated annual costs for England associated with the SBL care bundle.

https://doi.org/10.1371/journal.pone.0250150.t005

A series of one-way sensitivity analyses were conducted to explore the impact of assumptions made to estimate costs. Selected sensitivity analyses are shown in Table 6 below (see S1 Table for all alternative assumptions explored). The assumption which reduced the estimated cost by the largest amount (-£14,123,076) was that only half of the increase in the number of inductions of labour observed were associated with the SBL care bundle (i.e. an increase of 10% rather than 20%). Including the cost of staff time to complete training courses associated with Elements 2 and 4 of the SBL care bundle increased the estimated cost by the largest amount (+£30,807,840).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Alternative assumptions for costs associated with the SBL care bundle (total costs across the whole of England).

https://doi.org/10.1371/journal.pone.0250150.t006

As reported above, the stillbirth rate was 0.84/1,000 births lower in the participating Trusts after SBL care bundle was implemented than before. Applying this to the number of births in England in 2015/16 (n = 666,052), an estimated 559 stillbirths per year may have been avoided. Using the primary estimate of the cost of the SBL care bundle (£93,116,650), this equates to £166,577 per stillbirth avoided. However, this figure should be interpreted with caution because it is not possible to determine the proportion of either the increase in secondary resource use or the reduction in stillbirths that are directly attributable to the SBL care bundle.

Strengths and limitations

This evaluation was strengthened by the adherence to a published protocol which used stillbirth as its primary outcome rather than a surrogate measure as often used in this area [11]. The evaluation was conducted in 19 different maternity units reflecting the diversity of UK maternity units and included approximately 14% of the births in the UK. The data presented here represent the major component of the Saving Babies’ Lives Project Impact and Results Evaluation (SPiRE project) which also included analysis of clinical practice guidelines and service-user and staff views to provide a holistic view of the impact of implementing the SBL care bundle. The results of the other components are reported elsewhere [21].

This study is limited by it’s before-and after study design which was employed pragmatically as the care bundle and its implementation were planned before the evaluation project was commissioned. The degree of implementation was self-reported by participating trusts which may not have accurately reflected the degree of implementation in the clinical service. Furthermore, the variation in levels, phasing, and strategies of implementation meant that there was no discrete moment within or across Trusts where a change in practice could be determined. The use of routinely collected data was both a strength and a limitation of the study. The use of routinely collected data allowed the project to achieve the necessary large sample size and to be undertaken in a comparatively short-time frame, but the use of these data were frequently challenging due to variation in codes and coding strategies employed by individual Trusts. Furthermore, routine data may include recognised coding errors in maternity health episode statistics [22], but as this would affect both the before and after time periods in this analysis the impact of this may be minimal. In some cases, this meant that data on certain items were excluded for some or all Trusts e.g. admission to neonatal unit and rate of hypoxic-ischaemic encephalopathy. This also meant that it was not possible to include costs (or potential cost-savings) associated with these items in the economic evaluation.

Meaning of the study: Possible explanations and implications for clinicians and policymakers

Reducing stillbirths (alongside reducing maternal deaths, neonatal deaths, and brain injuries) is an explicit priority of the UK government, with a target to reduce such events by 50% by 2030 and by 20% by 2020. Achieving this aim will require an annual rate of reduction for stillbirth of 3.3%, significantly greater than the 1.4% reported between 2000–15 [2]. If maintained, the annual rate of reduction described in the study sites (~5%) would be sufficient to achieve this ambitious target and represents the most rapid reduction in stillbirth rates in over 20 years and is on target to achieve the national ambition to halve the rate.

In common with earlier national care bundles in perinatal care (e.g. Necrotising Enterocolitis Care Bundle) this evaluation has demonstrated clear improvements in process outcomes, but it was more challenging to identify changes in element-specific clinical outcomes and relate these to the degree of implementation [23]. Although this evaluation was not able to determine whether implementation of the SBL care bundle or any individual element was associated with the observed reduction in stillbirth, the association is plausible. Detection of SGA infants, the strongest risk factor for stillbirth in the UK [24,25], improved by 59% and use of ultrasound scanning, the principal screening method for SGA infants increased by 24%. Importantly, increased frequency of ultrasound scanning was inversely associated with stillbirth, suggesting that a pathway of increased detection of SGA may reduce stillbirth. Unfortunately, causes of stillbirth, particularly fetal growth restriction, are poorly recorded in routinely collected data which prevents analysis to determine whether there has been a reduction in stillbirths from fetal growth restriction [26].

Unanswered questions and future research

Any argument proposing a link between implementation of SBL and the outcome measures must be weighed against the change in outcomes which were not associated with the SBL care bundle. For example, the 20% increase in elective Caesarean section, which is unlikely to relate to implementation of the SBL care bundle as it contains no recommendations about Caesarean section, is more likely to reflect changes in guidance from the National Institute of Health and Care Excellence published in 2011 [27]. In addition to changes in national policy, we were not able to identify specific initiatives which may have been implemented in participating Trusts contemporaneously. As the SBL care bundle continues to be rolled-out in UK maternity units further evaluation needs to be planned to identify potential causal effects on stillbirth and other clinical outcomes.

Stillbirths are relatively rare events in the UK therefore large, good-quality, datasets are critical to informing efforts to reduce the stillbirth rate. Health policy makers should consider the evaluation of healthcare programmes prior to their implementation, in order that the most robust evaluation possible can be incorporated to ascertain its effects on process and clinical outcomes. The AFFIRM study, which prospectively evaluated increased maternal awareness and staff education about RFM (i.e. Element 3 of the SBL care bundle) [28], demonstrated how a stepped-wedge cluster design can be used effectively in this context.

The analysis presented here provides information about the average effect of implementation of the care bundle across the 19 Trusts. However, we were not able to explore the reasons for the variation in levels of implementation between sites, thus further research is needed to explore the origins of variation seen in different settings. Analysis of clinical practice guidelines relating to the SBL care bundle in participating maternity units found wide variation in their quality [21]. An ethnographic study of a UK maternity unit with a strong safety record (which was not part of this evaluation) described six mechanisms in place to enhance safety including: clearly articulated, constantly reinforced standards of practice, behaviour and ethics and structural influences on mechanisms for safety (e.g. staffing levels and financial infrastructure) [29]. Thus, we hypothesise that influences accounting for variable implementation may include: human factors, leadership, availability of up to date, accurate clinical practice guidelines and sufficient resource to implement them. These merit exploration in future analyses and the information can then be used to inform future quality improvement programmes.

While the resources required to implement the SBL care bundle may seem large, they should be viewed from a national perspective. In 2014/15 approximately £2.5bn was spent in the NHS on maternity services (with 664,399 births) and in 2016/17 obstetric claims handled by the NHS Litigation Authority accounted for £4.3bn of new claims reported. Based on figures for 2014/15, the maternity tariff per birth was approximately £3,760; the direct costs associated with SBL care bundle are estimated to account for £5 of this with secondary costs estimated at £135 per birth. As the majority of Trusts participating in the study reported using GROW software to produce customised growth charts the costs to do so were included in the estimated costs, however it should be noted that this software was not specified in the SBL care bundle and that alternatives could be used. It was not possible to collect data regarding some important resources relevant to the implementation of SBL care bundle, for example the impact of incorporating additional elements into routine antenatal appointments without increasing the duration of appointments, or the costs associated with NICU admissions, and so it is acknowledged that there is uncertainty around the estimated costs. Future economic analysis of the impact of the SBL care bundle or similar initiatives should also consider the health and social care costs of stillbirth (estimated as £13.1M/year in 2018) [30], which are greater than for a live birth and also extend into pregnancies after stillbirth [31].