Evidence-informed capacity building for setting health priorities in low- and middle-income countries: A framework and recommendations for further research

The Environment level

Capacities of the health system. The capacity of a health care system to support priority-setting, and the associated capacities across the various levels of INNE, requires institutionalising priority-setting agencies at provincial, national and regional levels, ensuring that appropriate structures, processes and incentives are in place. There are several major examples of agencies responsible for setting health priorities in entire countries, or parts of them in the case of federal systems of governance (Dittrich & Asifiri, 2016). However, the analytical advantages and weaknesses of the various models are only beginning to be exposed, and their sustainability has yet to be fully tested (Dittrich & Asifiri, 2016). Prescriptive guidance would therefore be premature, and any useful guidance would unlikely be one-size-fits-all.

Nevertheless, factors that may support institutionalisation of explicit priority setting in the context of LMICs have recently been identified in a policy brief co-authored by members of HTA agencies belonging to HTAsiaLink, a regional network (Chootipongchaivat et al., 2016). Its recommendations (see Box 1) are based on the experience of seven settings: China, Taiwan, Indonesia, the Republic of Korea, Malaysia, Thailand and Vietnam. The authors identify five conducive factors for HTA development and provide a practical step-by-step guide, including a checklist for monitoring the progress of HTA introduction and development (Chootipongchaivat et al., 2016). Although the policy brief focuses on the use of HTA to inform coverage decisions under universal health coverage (UHC), these recommendations could also be applied to HTA in general resource allocation.

Any priority-setting frameworks that simply generate evidence of what works and represents good value for money are inadequate (Chootipongchaivat et al., 2016; Rutter, 2012). “Good value” policy options may have no direct bearing on financial protection, and the distribution of financial and disease burden, which are important issues for UHC (Voorhoeve et al., 2016). In addition, the “right” decisions, even when evidence-based, don’t always get implemented for a number of practical, political or other reasons. It may be entirely rational for policymakers to make decisions against evidence-based recommendations, if by that they suit their own political interests, for instance to win electoral support from the ‘median voter’ (whose particular concerns may differ from what would benefit the population on a whole) (Hauck & Smith, 2015). This underscores the importance of developing a robust, principled process that considers such constraints, within which explicit methods for evidence-informed priority-setting can be institutionalised (Chalkidou et al., 2016b).

When we refer to ‘institutionalising’ priority-setting and HTAs, we seek to emphasise the importance of developing accepted norms and rules, and sustaining effective working relationships between relevant policymakers and research institutions (Hawkins & Parkhurst, 2016; March & Olsen, 2008). Norms and rules based around notions of transparency, accountability, citizen engagement, openness, deliberation, and contestability are valuable beyond having intrinsic moral merits, because they improve both the quality and credibility of decisions arising from evidence-informed priority-setting (Culyer, 2012; Daniels, 2000). Relevant processes that should be built in when institutionalising priority-setting and HTA include (Culyer, 2012; Daniels, 2000):

These processes contribute to good governance in evidence-informed priority-setting, which enables it to become more resilient to vested interests and political change (Hawkins & Parkhurst, 2016).

At the Network level

Capacities of funders and development partners. Global funders and development partners, including supra-governmental organisations like the World Health Organisation (WHO) and the World Bank, have significant power in shaping health priorities at the country level in LMICs (especially in low-income countries) (Chalkidou et al., 2016b; Glassman & Chalkidou, 2012). This can operate directly through their purchasing or provision of specific health care interventions, delivery platforms, and investment into research and technical assistance activities related to the above; or indirectly through their role as setters of global standards and norms, for example with the iDSI Reference Case (Wilkinson et al., 2016) and WHO CHOICE (Chalkidou et al., 2016b).

Funders and development partners need the specific capacity to commission, receive, interpret and use HTA and priority-setting research to inform not only their own choices in global health, but also the global standards and norms which client countries look to. Health system strengthening efforts could also be targeted towards the multitude of stakeholders and capacity gaps identified here, with the broader objective of supporting effective, evidence-informed and sustainable priority-setting that is country-owned (Chalkidou et al., 2016b). There should be a shared understanding within and between funders, delivery partners and LMIC country partners of the goals or outcomes of aid investment, in terms of funding, research outputs and technical assistance. This shared understanding could take the form of a common theory of change, that is a framework outlining the preconditions, causal linkages and assumptions underlying the desired investment goals (Li, 2016).

At an internal iDSI Board meeting in Bangkok in January 2016, we asked four funder representatives who were present (from the Bill and Melinda Gates Foundation, UK Department for International Development, Rockefeller Foundation and the World Bank, respectively) what internal capacity-building they felt would be useful in their organisations in order to support priority-setting better. Three funders felt that their organisations should develop rapid response services for country partners requesting technical assistance, both in terms of being able to direct them to relevant and useful evidence sources as well as identifying international experts capable of providing immediate short-term technical support. The fourth funder reiterated the importance of having the capacity to use value for money in guiding investment decisions, pointing to the iDSI Reference Case (Wilkinson et al., 2016) and other ongoing efforts to incorporate components of HTA in the grant-making process.

The Node and Individual levels: Consumers of evidence

Capacities of policy and professional decision makers. There seems to be considerable variation in the extent to which both policy and professional groups possess the capacities detailed in Table 1, and there has been little research thus far that documents it (though see Hailey & Juzwishin, 2006, highlighting that the inability of policymakers to formulate appropriate questions risks diminishing the policy-relevance of HTA programmes). Routine follow-up and monitoring of impact of HTA research by decision makers as an integral part of evidence-informed priority-setting is rare, as is evidence of any matching training programmes targeted at developing such capacities among policymakers. Fundamentally, there needs to be political commitment among policy leaders to progress to UHC and use evidence and tools such as HTA to help achieve that aim (Li et al., 2016).

Capacities of health service managers. NICE in the UK engages service managers in their HTA processes to select healthcare interventions and clinical guideline recommendations at the national level (National Institute for Health and Care Excellence, 2015). Among clinical research and health services research in general, however, health service managers are rarely included. This is possibly both symptomatic of and perpetuating the phenomenon that most HTA has focused on comparing individual interventions, as opposed to service delivery platforms or different organisational modes for human resources (Morton et al., 2016). HTA may therefore not provide sufficient information on the broader financial and organisational implications of competing resource allocation strategies that health service managers need in order to make fully informed decisions (MacDonald et al., 2008). The use of HTA to support planning by local health service managers in the UK has also arguably been hindered because of the relative inaccessibility of, and concerns over the acceptability of, the specialist methods employed (Airoldi et al., 2014).

Irrespective of the scope and complexity of HTA, the practical implementation of evidence-informed policies and practices crucially depends the on managers’ ability to set and enforce clinical standards and gain local adoption of good practice in both primary and secondary care settings, arranging funding and bringing local communities along through supportive and constructive local engagement. In the UK the National Health Service (NHS) has a relatively well-established tradition of clinical governance (Scally & Donaldson, 1998; Swage, 2003), and routine performance measures of healthcare managers and providers now include how successful they are in implementing clinical governance (National Institute for Health Research, 2017). In LMICs on the road to UHC, the capacity of health service managers to understand the implications of evidence-informed developments, competing spending options, and of managing resources accordingly will require specific training and ongoing support.

Capacities of patients and the public. Setting priorities in health implies that some interventions and some patient groups will be covered and others will not. There is a risk that because of this, those who do not see themselves as privileged, along with their carers and supporters, lose whatever enthusiasm they may have had for developing UHC. Their continuing engagement, and understanding of the process and decisions, are important both morally and for the success of the strategy (Clark & Weale, 2012).

Patients and the general public need to understand the implications of policy and clinical decisions and of the decision-making process, the extent to which specific decisions are evidence-informed and represent efficient and ethical use of public or private money, and they need to participate with an active voice in decision shaping that affects their interests. Capacity development activities could include training of health workers, patients and the public in research projects in the field, and other forms of patient involvement (e.g. HTA appraisal panels, citizens’ juries) (Littlejohns & Rawlins, 2009) in conjunction with the development of tools to facilitate stakeholder engagement in priority-setting (Bolsewicz Alderman et al., 2013; Makundi et al., 2007; Weale et al., 2016). Any such tools will be context-sensitive, if not context-specific, and take into account the socio-cultural values and political environment of the country or region (Bolsewicz Alderman et al., 2013).

The Node and Individual levels: Producers of evidence

Capacities of academic institutions, researchers and research managers. Healthcare researchers for LMICs tend to regard research capacity development in terms of the acquisition of research skills (e.g. World Health Organization, 2015), mainly through masters and PhD programmes offered by major centres in high-income countries. They measure success in terms of the various kinds of training received and in authorship in so-called ‘high-impact’ journals, which are predominantly published in English. Equally important, however, is how skilled local research communities are in engaging with policy and professional end-users, discerning their decision-related needs for evidence, and identifying what is researchable, and in translating those needs into research projects and programmes that can be implemented locally (with or without assistance from elsewhere). For low-income countries the key lessons to be learned may lie not with high-income countries, but with middle-income countries.

Networks that link the research community to policy decision-makers, professional regulators and professional colleges (Ezeh et al., 2010), as well as institutional and personal relationships, all need to be explored further. These relationships exist to some extent in all countries but may not focus particularly on the development of strategic commitments to the provision of timely and relevant evidence and analysis, or their institutionalisation into established practices through standing committees, routine communication (e.g. electronic) and other standard operating procedures (Hawkins & Parkhurst, 2016; March & Olsen, 2008).

With respect to technical capacities for research in LMICs, while there are relatively abundant resources in public health and epidemiology (although Ezeh et al., 2010, highlighted particular gaps in Sub-Saharan Africa), there is an even greater shortage of skills in high quality economic evaluation that would enable research teams to offer evidence of cost-effectiveness to achieve better and more equitably distributed health outcomes (Doherty, 2015; World Health Organization, 2015). In Sub-Saharan Africa, there is also a shortage of competency in systematic reviewing, especially in reviewing designed to economise on the need for new research by appropriate and critical translation of results from previous studies (Doherty, 2015). There also exists limited networks between African institutions in terms of research collaborations in health economic evaluation; the collaborations that do exist tend to be with North American or European institutions (Doherty, 2015; Hernandez-Villafuerte et al., 2016). The significant health economic research activity, capacity, and capacity-building initiatives in related disciplines that already exist in South Africa suggest that it is well placed as a hub country for catalysing South-South collaborations with other African countries (Ezeh et al., 2010; Hernandez-Villafuerte et al., 2016).

A more comprehensive and strategic approach to capacity building might embody the following additional features (see also Ezeh et al., 2010):

Leadership, management and administration

Technical and research skills

Knowledge transfer and exchange

The Node and Individual levels: Knowledge brokers

Capacities of knowledge brokers. Knowledge brokers and knowledge brokering agencies are intermediaries between worlds of research and action (Lomas, 2007). Their role involves “all the activity that links decision makers with researchers, facilitating their interaction so that they are able to better understand each other’s goals and professional cultures, influence each other’s work, forge new partnerships, and promote the use of research-based evidence in decision-making.” (Canadian Health Services Research Foundation, 2003). Capacity building is part of their philosophy: for researchers to be able to do applied research and decision-makers to be able to use it (Lomas, 2007).

Knowledge brokers can push for improvements on the evidence-supply side, for instance by packaging it better and by disseminating it in a more organised way (Lavis, 2016). They can also work on the evidence-demand side, for instance by advocating for the creation of institutional mechanisms that privilege the use of research evidence and building capacity to find and use research evidence efficiently (Chalkidou et al., 2016b; Lavis, 2016).

To achieve all of this, knowledge brokers must have the capacity to understand the cultures of both the research and decision-making environments. They need to be able to identify the ‘right’ stakeholders from both sides, and achieve meaningful knowledge transfer between them. Stakeholders from the different environments include researchers and decision makers, government agencies and local hospitals, professional organisations and community workers, and so on.

Of particular relevance to LMICs, where capacities on both demand- and supply-sides may be sparse, is focusing capacity-building efforts on existing agencies or groups of individuals with some formal linkage between the research and decision-making circles, including those who themselves function as a research unit (for example, the technical unit within a ministry of health) (Li, 2016). HITAP in Thailand is a good example of an institution with dual function as a generator of primary research in health economics and health policy, and as a knowledge broker through HTA processes that convene stakeholders including policymakers, clinicians, and civil society (Jongudomsuk et al., 2012).

Capacities of media organisations and journalists. The ongoing claims of a finite budget made by different stakeholders lie at the crux of priority-setting in health, and in many countries the media wield significant power to influence how these claims are understood by the general public and acted upon by policymakers, an issue perhaps more important than ever in the so-called “post truth” era (Marmot, 2017). We mean media in the broadest sense, so we are including journalists and editors in TV and print and those who communicate primarily through electronic media such as Twitter and blogs, in particular those with a specialist interest in health, government policy, science, or development.

While the role of the media varies from country to country, there will be technical, political and ethical issues in priority-setting that are shared across settings (Briggs, 2016; Hauck & Smith, 2015; Kieslich et al., 2016; Rumbold et al., 2017). There will also be general principles and common challenges to overcome in understanding and communicating notions such as priority-setting, rationing and fair access to services, for example the fact that evidence-informed priority-setting decisions are made with the whole population in mind but will inevitably lead to winners and losers among individual patients. We are not suggesting any compromise to editorial independence or the need for journalists to hold key stakeholders accountable. Instead, the aim is to encourage a greater understanding of the complexity of the priority setting process and to enable better informed and impartial reporting.

Recommendations for further research

In Table 2 we propose a set of research recommendations addressing the capacity needs of different stakeholder groups in priority-setting, in order to inform any future capacity building strategy adopted by iDSI or other development initiatives. Given the focus on targeting different stakeholders, we also recommend that a tool for mapping relevant stakeholder groups be developed that can adapt to different national contexts (Li, 2016).

Capacity building should be a two-way process; those who engage in capacity building should also reflect on their own capacity development to ensure their activities have the impact desired in the short and long term (Itad & NICE International, 2016). iDSI has a Monitoring, Evaluation and Learning framework to track ongoing implementation, collect evidence of iDSI contributions to stated aims, enhance accountability to members, stakeholders and funders, and encourage ongoing reflection and learning (Li, 2016). In addition, iDSI and its core partners have subjected themselves to independent reviews in order to reflect on progress, achievements, and operational arrangements (Health Intervention and Technology Assessment Program, 2009; Itad & NICE International, 2016). A Mid-Term Learning Review has been conducted to ensure iDSI remains fit-for-purpose and help identify potential capacity gaps and how these can be addressed (international Decision Support Initiative, in preparation).