Headache service quality: evaluation of quality indicators in 14 specialist-care centres

Abstract

Background

The study was a collaboration between Lifting The Burden (LTB) and the European Headache Federation (EHF). Its aim was to evaluate the implementation of quality indicators for headache care Europe-wide in specialist headache centres (level-3 according to the EHF/LTB standard).

Methods

Employing previously-developed instruments in 14 such centres, we made enquiries, in each, of health-care providers (doctors, nurses, psychologists, physiotherapists) and 50 patients, and analysed the medical records of 50 other patients. Enquiries were in 9 domains: diagnostic accuracy, individualized management, referral pathways, patient’s education and reassurance, convenience and comfort, patient’s satisfaction, equity and efficiency of the headache care, outcome assessment and safety.

Results

Our study showed that highly experienced headache centres treated their patients in general very well. The centres were content with their work and their patients were content with their treatment. Including disability and quality-of-life evaluations in clinical assessments, and protocols regarding safety, proved problematic: better standards for these are needed. Some centres had problems with follow-up: many specialised centres operated in one-touch systems, without possibility of controlling long-term management or the success of treatments dependent on this.

Conclusions

This first Europe-wide quality study showed that the quality indicators were workable in specialist care. They demonstrated common trends, producing evidence of what is majority practice. They also uncovered deficits that might be remedied in order to improve quality. They offer the means of setting benchmarks against which service quality may be judged. The next step is to take the evaluation process into non-specialist care (EHF/LTB levels 1 and 2).

Background

Headache disorders are a major public-health problem. They are common, give rise to widespread ill-health [18] and are the third highest cause of disability worldwide [16, 20]. Through lost productivity, they impose substantial financial burdens on individuals and society: in Europe their annual cost exceeds EUR 100 billion [5]. They are largely treatable, but effective care fails to reach most people worldwide [16, 22].

Although the scope, character and content of headache practice have changed markedly during recent years, the principal driver has been the introduction of new drugs. While the need for better education of medical professionals has been authoritatively proclaimed as a prerequisite and high priority for improving headache care [22], adequacy of care in its broadest sense has not tended to engage social awareness. This must change: the concept of service quality needs to take centre-stage in headache care, and become the driver for both headache centres and health-care providers (HCPs) in primary care, who manage most patients with headache [14].

This study continues the service quality evaluation collaborative project between European Headache Federation (EHF) and Lifting The Burden (LTB), within the Global Campaign against Headache [10, 11], which is conducted by LTB in official relations with the World Health Organization (WHO).

Quality assessment of headache care requires contributions from three parties: HCPs, other health-care workers and patients. As a collaborative group of health-services researchers and headache specialists, we first defined quality in the context of headache care [7], then formulated a set of quality indicators based on this definition that might be applied to headache care across countries, cultures and settings [8]. The ultimate purpose was that inadequacies in headache care worldwide might be recognized and rectified. Taking account of the attributes of care and of criteria for what constitutes good care, we proposed a multidimensional definition of quality:

“Good quality headache care achieves accurate diagnosis and individualized management, has appropriate referral pathways, educates patients about their headaches and their management, is convenient and comfortable, satisfies patients, is efficient and equitable, assesses outcomes and is safe” [7].

In a pilot study in two specialist headache centres in Germany and Portugal, we evaluated quality indicators, and related assessment instruments, in nine domains aligning with this definition (see Table 1) [3]. We interrogated HCPs and patients, finding that the enquiry was feasible and the questionnaires were easily understood by both groups, without being unduly time-consuming. The results from the two centres were intuitively meaningful, and comparable despite their very different settings (one university hospital-based, the other private hospital-based) [3].

Table 1 The nine domains of quality in a headache service (from reference [8])

The present study extends this evaluation Europe-wide, and involves a range of competencies between leading academic specialist headache centres with advanced multidisciplinary care delivered by headache specialists (level-3 according to the EHF/LTB standard [14]) and non academic centres with ambulatory care delivered by physicians with a special interest in headache (level-2 [14]). The aims were to demonstrate wider acceptability of the instruments and feasibility of use across countries and settings, and their fitness for purpose and utility as headache service quality indicators.

Methods

Ethics approval

Each participating centre applied for ethics approval according to local regulations. Since the primary purpose of the study was service quality improvement, in some countries it was deemed to fall outside the scope of research ethics review. Informed consent was obtained from all study participants (service staff and interviewed patients) regardless of whether or not ethics approval had been required. Data relating to individual patients were collected anonymously and held in accordance with European data-protection legislation.

Study settings and participating centres

We sent invitations to 23 secondary (level-2) or tertiary (level-3) headache clinics in 14 countries, selecting centres known to us as providing expert care and likely to be able to participate. One German clinic declined (not wishing to spend time applying for ethics approval), as did another in Germany, one in the USA, one in Serbia and six in the Netherlands for unknown reasons. Fourteen clinics in 12 countries of Europe participated over 8 months between February and September 2014. They are described in Table 2.

Table 2 Characteristics of headache centres

Study participants

The subject of study at each centre was the quality of service. Service staff (doctors [including trainees, where appropriate], other HCPs [nurses, psychologists, physiotherapists, others], the service manager [the person responsible for ensuring the service was properly run and maintained, who might be the clinical chief or a health-service manager] and the appointments administrator [usually a nurse or secretary]) and patients were witnesses; they were therefore the study participants, although not themselves objects of study.

The patient participants at each centre were a prospective consecutive sample (n = 50). In addition, information was acquired from the records of a retrospective random or consecutive sample of 50 patients other than those seen prospectively.

Study instruments

There were five questionnaires: one each for doctors, other HCPs, the service manager, the appointments administrator and patients. The last took the form of an exit questionnaire. This and the questionnaire for other HCPs (when necessary) were translated into the local language(s) by the local investigators according to LTB’s translation protocols [6]. Questionnaires are attached in Additional files 1, 2, 3, 4 and 5. In addition, some items of information (see Table 3) were extracted from patients’ or service records in a retrospective review by a staff member of the service using a sixth instrument. Table 3 shows the methods of implementation of quality indicators.

Table 3 Methods of implementation of quality indicators

Procedure

At each centre, all staff completed the relevant survey questionnaires, doctors and other HCPs responding anonymously. Patients received their questionnaires (with age, gender and principal diagnosis pre-recorded) from nursing or administrative staff as they exited their consultations. They were encouraged to communicate their experiences and points of view in the questionnaire, completing it anonymously and returning it before leaving. Data extraction from the charts, records or electronic database of the retrospective sample was performed by a doctor, nurse or research assistant, under the supervision of the local principal investigator, using the study instrument provided for this purpose.

Data management and analysis

Data were entered locally in each centre into spreadsheets provided, and the completed spreadsheets transferred to the central data collection centre (Institute for Medical Informatics, Biometry and Epidemiology, University Hospital of University Duisburg-Essen) where they were merged and analyzed by Sara Schramm. Demographic and clinical data were provided as numerical values and summarised as percentages or mean values with standard deviations (SDs). Analyses were descriptive only, with no statistical tests performed. Analyses were completed in Microsoft EXCEL 2010.

Highlighted problems and feedback from centres

After data analysis we sent the findings, highlighting any problems indicated by them, to all participating centres for explanation or other feedback. Specifically, each centre was asked for a short commentary on these problems and to explain apparent quality deficits (citing, where apropriate, mitigating factors or obstacles to good-quality care built into the local health-care environment). These responses are presented with commentary.

Results

Table 4 shows the characteristics of the 14 study centres and their participating patients. The date of completion of all items was between 3 February 2014 and 17 November 2014 with one exception: the records review of Russia-Moscow University was initially missing, and completed later between 2 and 5 October 2015. With different structures, the centres all had one manager but between one and 7 doctors, 0 and 16 nurses and other HCPs and one and 8 secretarial or administrative staff. Centres returned questionnaires from 30 to 58 patients, with mean age ranging from 34.9 years in Russia-Moscow University to 46.3 years in Denmark-Copenhagen. Mean duration of the presenting headache disorder ranged from 6.8 years in Georgia-Tbilisi to 17.6 years in Norway-Trondheim. The spectrum of diagnoses was very different across centres (Table 4). Each centre analyzed between 34 and 58 retrospective records.

Table 4 Characteristics of participating centres and patients

On a practical level, the questionnaires in 10 different languages were reported as easy to apply and were understood and accepted by both HCPs and patients. They were not unduly time consuming. None of the specific enquiries caused or led to difficulties. Evaluation of each clinic according to the quality indicators is shown in Table 5. Findings are summarised below in domains. Problems highlighted at the centres and explanatory commentaries of each centre are listed in Additional file 6.

Table 5 Results of the questionnaires (% of positive answers)

Domain A. Accurate diagnosis is essential for optimal headache care: The temporal profile of the headache was recorded in most clinics and diagnoses were made according to current International Classification of Headache Disorders (ICHD) criteria in more than 90% of cases in all headache centres. Not all centres had a system of working diagnosis at first visit, definitive diagnosis at first or subsequent visits, and review of the diagnosis during later follow up, which was explained by various organizational restrictions on opportunities to offer follow-up visits. Diagnostic diaries were available in all centres but Norway-Trondheim.

Domain B. Individualized management is essential for optimal headache care: Not all centres reported a formal triage system to expedite appointments in cases of perceived urgency, but this applied to disorders such as cluster headache rather than secondary headaches with serious underlying disorders. Mean time allocated to patients’ visits (according to patients’ reports) ranged between 12 and 31 min. Whereas the majority of the patients were satisfied with this allocated time, a high number of HCPs would have preferred more time per visit. In many centres there was no access route to psychological therapies and no instrument for disability assessment. In 4 centres, not all patients received follow-up who needed it because of the organizational barriers mentioned above. Follow-up diaries were available in almost all centres.

Domain C: Appropriate referral pathways are essential for optimal headache care: Our study identified a need for better referral pathways in some countries.

Domain D. Education of patients about their headaches and their management is essential for optimal headache care: Information leaflets were not available at all centres, although there were information leaflets in English freely available on the EHF-website [2] and leaflets in six European languages available from LTB [4]. Nevertheless, the great majority of patients expressed satisfaction with the information given by the doctor. Patients received appropriate reassurance in almost all cases.

Domain E. Convenience and comfort are part of optimal headache care: More patients than HCPs considered that the service environment was clean and comfortable; almost all patients felt welcomed. Waiting times in the clinics varied quite widely around means of 4–43 min, but were unsatisfactory for only a sizeable minority of patients. Regardless of this, up to 100% of HCPs were dissatisfied with the waiting time.

Domain F. Achieving patient satisfaction is part of optimal headache care: Overall satisfaction with their management was expressed by most patients, apparently regardless of waiting time and time allocated to their visits.

Domain G. Optimal headache care is efficient and equitable: Half of the clinics had protocols to avoid wastage of resources. Running costs were calculated in half of the clinics also. Only three clinics were able to offer equal access to headache services for all who might need it.

Domain H. Outcome assessment is essential in optimal headache care: Outcome assessment, also relevant to evaluation of cost-effectiveness in future health-care plans, required improvement in many of the centres. Quality-of-life measures in particular appeared to be underused.

Domain I. Optimal headache care is safe: Formal protocols to ensure reporting of serious adverse events were lacking in many centres.

Discussion

This study was not planned primarily to assess service quality in the 14 centres but to evaluate the quality indicators and a range of related instruments by which such assessments might in future be conducted. Europe-wide, we were able to implement the instruments in 10 different languages, applying 26 indicators. The centres were diverse in level of care, setting and structure and operated within different cultures and health systems. Implementation was readily acceptable and all centres were able to collect data quickly and efficiently in the nine domains deemed important for quality evaluation [3, 8]. Across centres, findings were comparable. Our study showed common trends in practice, and what was majority practice, and could therefore guide the setting of benchmarks against which quality might be judged. At the same time it uncovered deficits in individual centres, indicating need for more systematic use of diagnostic diaries and disability and quality of life instruments, and revealing the restrictions on opportunities for follow-up visits. In these ways our study confirmed that the indicators could be regarded as a guide to improve quality, were fit for purpose and had utility.

In some centres, the findings reflected local system difficulties. The structures of the centres were different: there were university hospitals and both public and private clinics with varied financial resources and differently structured health care systems. Many centres had no control over equal access. We saw that some centres had problems with follow-up: many specialised centres and universities were operating in one-touch systems, as a rule seeing their patients only once before sending them back to peripheral doctors with advice. These centres had no possibility to control the success of treatment; worse, from the point of view of audit and quality maintenance, they might never know whether their recommendations led to success or failure. Such systems do not inevitably lead to suboptimal care, but they certainly do not promote quality of care. Restrictions imposed by operating environments and resource limitations often lay behind absence of referral pathways and lack of access to some disciplines of care such as psychological therapy. The development of benchmarks based in part on practice in other centres may be an evidence-based means by which these restrictions can be lifted, if this requires action at political level.

Other deficits appeared to be within the control of centres. Diagnostic diaries, demonstrated to be very valuable both in guiding diagnosis and in setting treatment plans [19], were not universally available. Failure to have and make use of instruments for disability assessment — although disability is a major consequence of headache disorders and a range of well-validated disability instruments exists [1, 9, 12, 17, 21] — appears to be both necessary and easy to remedy. The same may apply to quality-of-life assessment, a problem in almost all centres, although this is more relevant to outcome assessment during follow up than to the formulation of treatment plans at time of presentation. The mindset of most of the centres appeared to focus on treatment of patients’ symptoms.

These findings — demonstrating to centres where standards are not met and improvements are needed — are both a call for and a guide to action. We saw differences between newer and older centres, the former still working on the implementation of standards. There were different levels of knowledge and experience among HCPs: some centres were run only by permanent headache-experienced HCPs, others — especially those with training roles — made use of HCPs with less experience and relatively high turnover. In some centres HCPs were not familiar with the overall structure of their service, or aware of the existence of special protocols or processes. Here our findings are a guide for extra teaching in these centres.

More patients than HCPs considered that the service environment was clean and comfortable. There was generally more dissatisfaction with waiting times and times per visit among HCPs than patients. Overall satisfaction with their management was expressed by most patients, emphasising the rewards that a good headache service can bestow on its patients (and, more broadly, all society, which bears a substantial part of the burden of headache [15]). These findings raise the question: What are the expectations of patients? Neither waiting time nor time allocated to patients’ visits appeared to influence patient satisfaction. A thorough evaluation of service quality may need to look more closely into the reasons for satisfaction or dissatisfaction of patients: while the original formulation of the quality indicators took patients’ expressed preferences into account [7], there may be intercultural differences. Most patients suffering from headache who receive timely and good-quality health care can expect this care to be effective, restoring quality of life [12], but good quality in health care has not been the automatic result of the marked changes in scope, character and content of headache practice and care that have occurred during the last years. Quality of care has not been much subject to social awareness or interest: our study, in the context of the collaborative EHF/LTB project of which it is part, is a step towards bringing headache service quality centre-stage. This takes importance from the fact that there has been no similar initiative preceding it.

The strengths of our study are its international scope, inclusion of many diverse centres within different cultures and interrogation of all three interested groups: HCPs, other health-care workers and almost 700 patients. The principal limitation was that we did not collect any information about the quality of the treatment itself. In a full quality evaluation, external expert reviews would check additional quality indicators relating to treatment in each centre by retrospective examination of the records [8]. However, we did not have the resources to undertake this part.

It is a matter for further studies to set benchmarks. A key pertinent question is: does majority practice correctly indicate what the benchmark should be, or might acceptance of majority practice in quality indicators crystallise suboptimal practice? In relation to this, future studies might question whether availabilty of referral pathways to psychological therapies and use of quality-of-life assessment are essential to quality; neither is supported by universal practice or evidence of cost-effectiveness. A second key pertinent issue is whether the same benchmarks are appropriate at all levels and in all settings, on the basis that quality is an invariable construct. Also in future studies, the following might be subjects of especial scrutiny:

• Do waiting-list times for appointments actually reflect urgency of need?

• Do treatment plans follow evidence-based guidelines according to diagnosis?

• Are patients not over-investigated (special investigations of concern include MRI, CT, EEG, Doppler, evoked potentials, skull and neck xrays)?

• Are patients not over-treated (over-treatment may mean excessive use of drugs likely to induce medication overuse headache (MOH), overdosage with potentially harmful drugs such as ergotamine or steroids, use of prophylactics for infrequent headache, use of prophylactics for the wrong diagnosis, or use of non-evidence-based treatments that are unlikely to be effective and may jeopardize safety)?

Conclusions

In conclusion, this Europe-wide study was the first study anywhere to evaluate headache care quality indicators across a culturally diverse multinational range of settings. It showed broad commonalities while uncovering deficits that might be remedied, and demonstrated that the indicators were fit for purpose and could be a guide to quality improvement, at least in specialist care. It highlighted lack of systematic use of diagnostic diaries and disability and quality-of-life assessment instruments, and restricted opportunities for follow-up visits. It showed what was majority practice and, whether based on this or not, can guide the setting of benchmarks against which quality may be judged.

The next step is to take the process into non-specialist care, since this is where most headache patients are and should be treated [12, 22]. The finally-agreed quality indicators should not themselves be varied when taken into primary care, even though the benchmarks might be different.