Coherence: Understanding the purpose of the MedsST and its data.

Regarding the views on the purpose of the tool, the study analysis highlighted that a common understanding existed, concerning the rationale for measuring medication safety for learning and improvement. However, in order to have confidence to engage with the MedsST implementation, NHS staff required clarification on the operation of the tool, such as; how data should be collected and by whom, and available support networks that could be used to facilitate this. All participants were in agreement that the MedsST’s purpose was to aid improvement of medication safety by enabling organisations to quantify medication safety issues and providing a “base-line” (P10, Nurse, EA) to monitor improvement. Specifically, it was used to identify the most problematic hospital wards and work with them to improve medication safety.

Most organisations had previously used yearly audits to measure medication safety. Staff with training or passion for quality improvement truly understood the benefits of monthly medication safety data, as opposed to yearly data from the traditional medication safety audits collected by most organisations.

Only one organisation had collected monthly medication safety data prior to implementing the MedsST, using an internally developed tool. The MedsST lead from this organisation reported that their medication safety data tool had been replaced by the MedsST, because of its national focus, and the ability to use the learning they had gained from their previous tool to contribute to the MedsST’s development, as this MedsST lead had joined the steering group who were leading the development of the MedsST. The MedsST lead preferred the previous tool, due to greater data “granularity” (P12, Pharmacist, EA). However, the MedsST user in the same organisation preferred the MedsST over the previous tool, because it involved nurses, saved time and provided immediate feedback for wards.

There was a clunky internal system before that resulted in our lead pharmacist having to plug away hours and hours and hours of data collation, and pulling together and feeding that back to us. That’s clearly not a robust way…(and the Medication Safety) Thermometer for our organisation is brilliant…it has slightly made our data collection better, I’d argue. P13 (Nurse, EA)

Participants reported an initial lack of understanding about how data could be used for improvement, particularly in LA organisations, where data collection was initiated prior to gaining a full understanding of how it should be used. Some participants felt strongly that the MedsST should not be used for pin-pointing individuals and for staff to have “the finger pointed at them.” (P3, Clinical Auditor, LA). Nonetheless, one pharmacist (P2, LA) stated that MedsST data had been used for pin-pointing poor practice of nurses and subsequent performance management. Most nurses agreed with participant 2 that MedsST data should be used for monitoring performance of nurses and that practice cannot be completely “blame-free”, as certain “stupid” individuals could cause errors; however, they also believed that errors are actually caused by system problems that need to be addressed by supporting individuals. Although national online guidance specifically states that the tool has not been developed to blame individuals [12], the study analysis highlighted that cognitive dissonance may exist regarding whether staff believe medication errors are due to specific individuals, or system problems. Although it was stated that errors may be due to individuals, it was also stated that the errors individuals make are usually due to system problems.

It can't be completely blame-free, because obviously some people are just stupid. But what you're not trying to do is beat them with a big stick, and look for trends, because usually when an error occurs it tends to be the system and not the person. P9 (Nurse, EA)

Although more junior staff understood the purpose of the MedsST was to collect medication safety data, they were unsure about how data were actually used. Lack of coherence was reported to be a problem in primary care, where data were mainly collected by junior staff. One MedsST lead from primary care believed that junior staff may not understand the value of MedsST data and reported a culture of “looking for mistakes” (P4, Pharmacist, LA), rather than looking for harm (regardless of how it occurred), which may have led to missed learning opportunities.

I wonder if the people there are quite junior…(and) are just doing what they are told by senior staff and…implementing it (MedsST)…not really thinking about the value of it…and thinking, 'no that's not an error, we haven't done anything wrong'…I tried to explain…it’s not just about error. P4 (Pharmacist, LA)

Although now based in primary care, Participant 4 had previous secondary care experience and quality improvement training. Therefore, they felt the culture of looking for errors, rather than harm, may be more predominant in the primary care settings, where there may not be organisational readiness for quality improvement tools, such as the MedsST.

Cognitive participation: Engagement with the MedsST and its data.

In order for healthcare staff to engage with the MedST, they had to have ownership of their organisation’s medication safety, by being involved with for the medication process, for example, the clinical auditor (P3) was not involved with the medication process and showed less engagement as reported below. Furthermore, in organisations where staff displayed coherence with wider quality improvement projects, greater engagement and organisational readiness for implementing the MedsST was reported, and the use of the MedsST appeared to have “normalised” into routine practice more easily [14]. Involvement with the MedsST strengthened medication safety ownership and led to improvements in participants’ own practice.

When I do come across a drug chart that hasn't been dated, I get quite frustrated because at the end of the day it is trust policy…with me having more involvement with this (the MedsST), I am really aware of it now. P5 (Pharmacy technician, LA)

Although difficulties with engaging ward senior management were reported, their involvement with data collection led to greater trust in and ownership of medication safety data.

I think if I’m the ward sister and someone tells me…’You’ve got 10% dose omissions’ and they’ve done the audit, it’s useful…but if I’ve actually done it (collected MedsST data), then I know that it was John in bed one and Barbara in bed three. P13 (Nurse, EA)

It was evident from the data that strong support networks, internally and externally to organisations, seemed to be fundamental for impetus for improving medication safety. In particular the formation of the Medication Safety Officer (MSO) network in 2014 [15] provided a helpful support network for MedsST leads who were also the MSO for their organisation (an allocated member of staff to support local medication error reporting and learning [15]). Although external support did improve impetus for using data for improvement, this impetus was decreased if there was a lack of support internally, particularly from senior management (at ward and organisational management levels). Where senior management staff displayed a lack of ownership of the MedsST a knock-on effect could occur, where frontline staff would show less engagement. Furthermore, management staff had to be “pro-active” (P9, Nurse, EA) to ensure that ward staff were aware of the MedsST data, and for further improvement work to be conducted. Lack of engagement from management staff was associated with decreased awareness of the MedsST within organisations and, therefore, lack of use of data for improvement. It was reported that more junior staff, such as pre-registration pharmacists, collecting data displayed less engagement, feeling it was “pushed” onto them.

As a group we feel like no-one wants to do it (collect data), so they just pushed it towards the pre-reg (pre-registration pharmacists), we won’t complain, we’ll just do it…(but) it would make more sense for a technician…to collect the data because they’re doing the meds rec (medicines reconciliation) every day. So they’re familiar with the chart, they’re familiar with the patient. P15 (Pre-registration Pharmacist, EA)

As mentioned previously, the Clinical Auditor (P3, LA) who was interviewed was the only participant who was not directly involved with medication dispensing or administering in the rest of their work, and reported that they had been “borrowed” by the pharmacy department to help collect data. Therefore, they demonstrated a lack of ownership of medication safety improvement and did not feel inclined to view or act on the data they had collected. Furthermore, their lack of involvement in the medication process, led them to feeling like a “lonely worker” who was not part of “the bigger picture”. Some MedsST leads also felt unsupported in improving medication safety, if they had no one else in their teams.

“In terms of governance and medication safety…I'm a bit of a one-man band…because…There isn't any other people (in my team)” P1 (Pharmacist, LA)

This feeling of lack of support had detrimental effects on impetus for improving medication safety, some MedsST leads had overcome this lack of support by forming or joining external medication safety networks, where they could learn about how to use the MedsST and its data for improvement. The support networks were often developed by pro-active MedsST leads, and enabled wards, organisations and regions to share innovative methods of using data for improvement, in addition to learning how challenges with data collection could be overcome. In most organisations the MedsST lead was also an MSO, however, in one organisation the MSO was not involved with the MedsST and the role of MedsST lead had been offered to a pharmacist with personal interest in quality improvement. This participant had unsuccessfully attempted to contact other organisations to learn how they had implemented the MedsST in primary care settings. Without internal or external support networks, the MedsST role had proven burdensome for the pharmacist who felt unsupported and that they were not being “listened to” (P4, Pharmacist, LA) about how the MedsST data could be used for further improvement work.

I am very aware that it shouldn’t really be my role…and I keep telling them that…“Yes I can implement this for you, I can tell you what the problems are etcetera, but you need to take it on”. P4 (Pharmacist, LA)

Many organisations had not experienced the benefits of a multi-disciplinary approach, as described by other participants, as nurses were not involved with data collection. Nonetheless, even participants from organisations where only the pharmacy staff were collecting data, believed that measurement and improvement of medication safety required a multi-disciplinary approach. It was generally reported that senior nursing staff said they did support the use of the MedsST; however, they did not show this support and had often used “lack of capacity” (P6, Pharmacist, LA) as an excuse for not allowing nurses to be involved with MedsST data collection. Furthermore, it was highlighted that involvement of nursing staff at ward level was important for fostering multi-disciplinary ownership of medication safety.

People see pharmacy…(and think) drugs. But…as nursing staff, we give them…They (pharmacists) don't actually administer them so there's no point (pharmacists) coming along thinking, ‘I'll fill all this and I know what we're doing’. Yes, it's dead easy for you to fill it in, but then there's no ownership at ward level and then the nursing staff don't get to know what's going on, they don't actually see it. P9 (Nurse, EA)

Generally, there was a lack of cognitive participation with Step 3, which involves a multi-disciplinary huddle to ascertain whether medication related harm has occurred, and the majority of organisations were not using it. Participants reported difficulties with gathering the pharmacist, nurse and doctor for a multi-disciplinary huddle. However, some organisations, despite facing similar staff and funding issues, reported that they had no issues with performing the huddle for Step 3 once people understood that it is a simple conversation between the nurse and junior doctor looking after the patient and the pharmacist collecting data.

I think the term MDT (multi-disciplinary) huddle has made people think that that's some kind of super 'I need to get it sorted out by e-mail, I need to get the consultants in' and no, that isn't like that, and that is why I say,… ‘Look, just grab the junior doctor, grab the primary nurse and see if they agree, and then put it in the incident system. If that needs investigation then we might need to get the consultant in and we might need to get the ward manager in’. P8 (Pharmacist, EA)

Mixed feelings were displayed about performing MDT huddles in primary care settings, the main benefit highlighted was the team-working between the pharmacist and a patient’s GP to review a patient’s safety in a novel way. However, there were practical difficulties with contacting the GP and making them aware of why they were being contacted.

Initially they (the GPs) started off (saying) ‘what the hell are you ringing me about’…but when I explained it (the MedsST) to them, yes they could see that there was a value and a point to it. So yes, that was positive. P4 (Pharmacist, LA)

Primary care NHS staff were generally supportive of the concept of the MedsST, but believed some questions were not applicable for their setting, and that using an alternative community version (version 16b) would not improve the issues faced. Of the four high risk drugs monitored using MedST, only insulin had triggered MDT huddles in primary care settings, leading to primary care organisations reviewing the suitability of these triggers. One primary care organisation had stopped using the MedsST for this reason, as reported by a participant from this organisation’s associated secondary care organisation.

The uptake in community (primary care settings) is really poor. We've stopped doing it in community because…what was the point in doing it if we weren’t detecting anything? Therefore, we are clearly asking the wrong questions. P8 (Pharmacist, EA)

Collective action: Actions taken to normalise use of the MedsST into routine practice.

Many activities were reported to have been undertaken to scale up implementation of the MedsST within organisations, including increasing the number of wards data were collected from, and securing additional funding and staff for this. Non-financial incentives existed for staff for their involvement in collection of data, such as protected time for overseeing data collection and analysis, involvement with related research projects and attendance at medication safety meetings.

Local financial incentives for organisations to use the tool, had often been organised at the suggestion of the MedsST lead to the local Clinical Commissioning Groups (CCGs) who set financial targets for healthcare organisations in England [2]. The introduction of financial incentives were referred to by participants as a “turning-point” (P6, Pharmacist, LA), and had led to management staff giving the MedsST data “greater respect” (P8, Pharmacist, EA); and more senior staff involvement with data collection to ensure accuracy of data. Although it was difficult to involve senior staff, as they already had high work-loads, doing so had led to increased ownership and passion for improving medication safety and, therefore, supporting analysis of MedsST data.

After…CQUIN (payments became available), obviously we had reports to do, every quarter we had to say why we hadn't achieved, there were penalties and no financial gain and that's probably where the turning point was and because of that I started really looking at the data. P6 (Pharmacist, LA)

Although financial incentives were reported to help drive data collection and scale-up of MedsST use, it was important to MedsST leads that they worked with local CCGs when introducing financial CQUIN payments, to ensure targets were realistic.

The problem is CCGs get hold of something and they don't actually understand it, they just tell us to use it. So we have a potential CQUIN (financial incentive target) with one of the CCGs and my report has to actually…tell them how useful (the MedsST) is…and how much time it takes. P4 (Pharmacist, LA)

Locally commissioned financial incentives appeared more successful than regionally commissioned financial incentives, and there was general agreement that national financial incentives should not be introduced. However, large-scale implementation nationally was described as a “great thing” (P2, Pharmacist, LA), but it introduced various complexities in different wards, organisations and regions, especially if they were still unsure of how to use the data for improvement. Participants discussed their organisation’s plans for scaling up use of the MedsST to all wards, which had been delayed multiple times, due to resistance from senior and ward level staff. Where organisations had successfully scaled up use of the MedsST to all wards, there was increased engagement of staff, and the MedsST had been embedded into routine practice to the point that it 'disappeared' from view (i.e., it was normalised) [14]. By contrast, in organisations where only some wards were collecting data, there was increased ward level resistance to implementation and by clinical champions (frontline users who take action to forward the implementation process) used personal friendships to overcome this.

(The ward sister) was really against doing it and she used to rant and rave at me every time we talked about rolling it out… I said, 'Look, I know (you don’t want to do it). But, you’re going to do it for me anyway aren’t you?’, and she said, “Yes because you are my friend, but otherwise, no, I wouldn’t”. P9 (Nurse, EA)

Although communication between MedsST leads with ward staff was occurring in most organisations to engage more wards to initiate use of the MedsST, communication was not occurring between senior organisational management, MedsST leads, ward level management regarding actually using the data collected, hindering the impetus for improvement using the collected MedsST data. For example, in organisations where data were analysed, the feedback was e-mailed to wards and not all frontline staff accessed e-mails. Additionally, for feedback to be acted upon, strong leadership was required from ward managers to create impetus for further improvement work.

It has to be coming from a hierarchy saying, ‘We need to develop something’. (If you) send out an e-mail…only ward managers read it and…not every ward manager is 100% proactive. P9 (Nurse, EA)

Furthermore, there was a lack of education and training for healthcare staff about how to make best use of the data to inform quality improvement. Training was provided for data collection, but even this was problematic, with participants reporting a lack of funding and resources to train more staff. Most MedsST users had learnt how to collect data by shadowing other MedsST leads or users, and by using national guidance. One organisation had developed standard operating procedures in order to create a shared understanding between staff of how to collect consistent data, but these activities required extra funding. Many MedsST leads reported an inability to influence implementation to a greater extent, due to lack of funding, resources and support from senior levels who prioritised other areas of safety improvement. For example, it was reported that collecting data on electronic tablet devices (rather than paper-based) had halved data collection time; however, establishing the use of electronic tablets as a resource and gaining appropriate permissions presented challenges, and had taken 3 months in one organisation.

Reflexive monitoring: Reviewing use of the MedsST and embedding changes.

To successfully embed the MedsST into routine practice, staff had to review their experiences of implementation and adapt the MedsST to suit local circumstances as necessary. Use of data included reviewing, analysing and trying to learn lessons from collected data. Although all participants were aware that they could access MedsST data via a dedicated website; however, very few had done so. Reasons for not viewing or using data included: technical difficulties (such as the website crashing), believing another department would view and act upon the data (such as the Quality Improvement department), time and resource constraints, communication issues and staff feeling ill-equipped or supported to analyse data.

Mixed feelings existed about the usefulness of data presentation on the dedicated website. For example, one MedsST user disliked the run charts used to display data and indicated preference for a written summary of change, which would require more management staff input.

It would be nice to have a little summary to say, "This actually shows that 40% of patients have had their medicines" or something. Because… if you are not used to the…graph… (it’s) like "What's that line there for?" and "What does that mean there?". P9 (Nurse, EA)

Conversely, senior management staff preferred even more detailed displays, such as Statistical Process Control charts [16]. However, it is possible that frontline staff would not have the appropriate skills to interpret more detailed graphs, as they already reported difficulties understanding the simpler run charts, as indicated above. Staff who had quality improvement training or a personal interest, particularly from EA organisations, were more confident using MedsST data. EA organisations generally displayed better understanding and ownership of the MedsST, indicating that involvement of users with early development of an intervention can positively impact implementation of it.

Generally, the majority of participants did feel the run charts used to display data on the website were useful for visualising progress over time, and that the immediate access to these run charts was beneficial and time-saving, for example, in response to freedom of information requests.

We had a freedom of information request about insulins: ‘How (many) omitted doses have we had in the last quarter?’ Just like that, it was so brilliant, at a push of a button I could say, ‘Oh yes, I know how many insulin doses were missed out of this proportion of these number of patients’. P2 (Pharmacist, LA)

Where participants understood the data presentation, they successfully used run charts to monitor improvement, and identify patterns and trends. Furthermore, collecting data on a monthly basis had been vital for identifying and investigating certain trends, such as the impact of staffing shortages on medication safety in December every year, or the impact of system changes on medication safety (for example, the introduction of electronic prescribing). When investigating changes in data, input from ward staff was reported as vital.

She (the ward sister) actually gave me reasons of why there was peaks in drug omissions. She said, 'Around that time we had a lot of agency staff'…so that makes sense because agency staff obviously don't have that ownership (of good practice) that a permanent member of staff do. P6 (Pharmacist, LA)

In addition, having routine medication safety data allowed users to demonstrate improvements at ward level, for example, for judicial inquiries.

(We needed) to be able to prove … that things have actually improved since (an incident occurred). …(So I thought) I’ll pull all the data off…so long as it looks good then we will send it, if it doesn't look good, let's not send it. But it did show that the ward (staff) were actually making strides to improve. P9 (Nurse, EA)

MedsST data also allowed healthcare staff to identify specific areas of medication safety for further improvement work, for example, a medication omissions awareness project had been conducted by one organisation, as they had realised their rate of omissions was higher than the national average according to the MedsST data. Mixed feelings existed about using MedsST data for benchmarking and comparison between organisations, departments and wards. Some participants agreed it should happen; however, some participants felt that comparison data between organisations would be beneficial, but not between wards, as wards should be “working together” (P3, Clinical auditor, LA). However, other participants argued that comparison between different wards’ data did encourage wards working together by increased sharing of learning between wards. This culture of learning from each other was more apparent in EA organisations.

Data on wards was being reviewed by other teams both internally and externally, and MedsST data was encouraging people to ask each other, “How did you get it right?” P13 (Nurse, EA)

LA organisations had more recently started sharing lessons learnt from using the MedsST and related improvement projects, particularly between organisations. Although there are platforms, such as online forums, available for sharing information about how the MedsST has been used and received within organisations, some participants (particularly nurses) were not aware of them and felt there were communication issues between organisations.

What goes on here might be brilliant, but unless someone is shouting about it from the rooftops, other wards might not be able to get to know about it. P9 (Nurse, EA)

Communication between primary care MedsST leads was problematic and they reported a lack of awareness regarding how to contact their peers.

I do think there might be potential (in primary care) but…I would like to speak to other community trusts to see how they are using it…and have looked at redesigning the triggers. P4 (Pharmacist, LA)