Figures
Abstract
Background
Research has shown that the modes of leadership and management may influence health outcomes. However, majority of health leaders and managers in many low-income countries are promoted on account of clinical expertise. It has been recognised that these new managers are often ill-prepared for managing complex health systems. In response to this challenge, the Zambian Ministry of Health (MoH) has developed the Governance and Management Capacity Building (GMCB) Strategic Plan (2012–2016), whose overarching goal is to improve health sector governance and create an environment that is result-oriented, accountable and transparent. This led to the introduction of a new in-service leadership and management course, which has come to be known as the Zambia Management and Leadership Academy (ZMLA). This paper presents the results of an impact evaluation of the ZMLA programme conducted in 2014.
Methods
This was a cross-sectional mixed method study. The study targeted health workers, stakeholders and course implementers. ZMLA trainees were targeted to gain perspectives on the extent to which the programme affected levels of self-confidence resulting from knowledge gained. Perspectives were sought from both ZMLA and non ZMLA trainees to measure changes in the work environment. Stakeholder perspectives were collected from trainers and key informants involved in providing ZMLA training.
Results
On average, knowledge levels increased by 38% after each workshop. A comparison of the average self-rated scores from 444 management and leadership survey responses before ZMLA and after ZMLA training showed a significant increase in the proportion of participants that felt adequately trained to undertake management and leadership, from 63% (before) to 99% (after) in phase 1 and 43% (before) to 98% (after) in the phase II cohort. The calculated before and after percentage change for work environment themes ranged from 5.8% to 13.4%. Majority of respondents perceived improvements in the workplace environment, especially in handling human resource management matters. The smallest improvement was noted in ethics and accountability. Qualitative interviews showed improvements in the meeting culture and a greater appreciation for the importance of meetings. Shared vision, teamwork and coordination seemed to have improved more in work places where the overall manager had received ZMLA training.
Citation: Mutale W, Vardoy-Mutale A-T, Kachemba A, Mukendi R, Clarke K, Mulenga D (2017) Leadership and management training as a catalyst to health system strengthening in low-income settings: Evidence from implementation of the Zambia Management and Leadership course for district health managers in Zambia. PLoS ONE 12(7): e0174536. https://doi.org/10.1371/journal.pone.0174536
Editor: Saravana Kumar, University of South Australia, AUSTRALIA
Received: December 3, 2015; Accepted: March 11, 2017; Published: July 25, 2017
Copyright: © 2017 Mutale et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data were gathered as part of a USAID project and cannot be made publicly available in order to protect participant confidentiality. Data will be made available upon request to qualifying researchers by contacting the President of BRITE, Dr. Iris Mwanza at imwanza@briteinc.org.
Funding: The project was funded by the Zambia Systems Strengthening Project (ZISSP) as part of the M&E for the Project.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Health systems governance is currently a critical concern in many countries because of increasing demand to demonstrate accountability in the health sector[1–3]. Research has shown that the modes of leadership and management may influence health outcomes such as life expectancy at birth, child mortality, maternal mortality, and self-reported health status.[4]. A study conducted in Kenya demonstrated increased coverage of maternal and child health services after piloting a six month leadership training in 67 management teams in Kenya [5].
It has been acknowledged that majority of health leaders and managers in developing countries are trained health professionals (doctors, nurses, clinical/medical officers and pharmacists) who rarely have any training or experience prior to being offered managerial positions[6, 7]. New managers are often promoted on account of clinical expertise alone. They may be ill-prepared for their new responsibilities and may be expected to gain managerial capacities by learning on the job or through training [6]. In the case of Zambia, district health teams are led by newly qualified medical doctors whose curriculum emphasizes clinical skills, with minimal attention to management and leadership. Recently there have been calls to strengthen leadership in the health sector through both in-service and pre-service training [1].
While didactic educational processes can be useful for providing ideas, material and motivation, they are not sufficient. The extent to which leadership and management are integrated into the personal identity depends on the extent to which the skills and knowledge are integrated into daily practice. This is often reinforced through coaching and mentoring[1]. Well integrated management and leadership practices can translate into positive change in organizational culture and routine processes. This can lead to improved performance which for health can be both system and health outcomes [4, 8]. This is summarised in the following logic framework:
The logic model (Fig 1) combines leadership and management concepts from Donald and James Kirkpatrick's learning and training evaluation theory,[9] McLeroy et al (1988) socio-ecological model of behavioural[10] and work done by the Management Sciences for Health (MSH). The model suggests that through exposure to program activities (workshops, mentoring, case studies), program trainees should:
- Demonstrate increased knowledge
- Be more confident in carrying out management and leadership functions
- Experience improved job motivation
- Improve management skills and behaviours that increase daily work performance.
This conceptual framework informed the design and evaluation of the ZMLA Programme. It was expected that participants exposed to the ZMLA programme would show improvements in the areas of strategic planning, problem analysis, use of data in decision making, running meetings, supervising and motivating employees. As a result of these changes in participants’ work practices, programme performance/service delivery and the work climate at participants’ work sites should also improve. By improving programme performance and service delivery, it was anticipated that the ZMLA programme would ultimately benefit the general population through access to better health services [11]. Complementing the logic model was the McLeroy et al (1988) socio-ecological model of behaviour change, which posits that behaviour is determined by the following factors:
- Personal factors: Characteristics of individuals, which include knowledge, attitude, behaviour, self-perceptions
- Institutional factors: These include social institutions, rules, and other organizational characteristics
- Community factors: Relationships among organisations, institutions, individuals, cultural systems, beliefs.
- Public laws and policies: For example national policies
It was theorised that improved skills and adoption of good practices would lead to improved workplace environment, improved performance and service delivery. This model was adopted as it captured the activities and aims of the leadership training course presented in this paper.
In view of the current leadership crisis in the health sector, the Zambian Ministry of Health (MOH) had developed the Governance and Management Capacity Building (GMCB) Strategic Plan (2012–2016), whose overarching goal was to improve health sector governance and create an environment that is result oriented, accountable and transparent. This led to the adoption of a new in-service leadership and management course, which has come to be known as the Zambia Management and Leadership Academy (ZMLA). The course has been adapted for the Zambian health system managers. The development of the course was supported by the Ministry of Health (MOH), Ministry of Community Development, Mother and Child Health (MCDMCH), BroadReach Institute for Training and Education (BRITE) and the Zambia Integrated Health Systems Strengthening Project (ZISSP). The course was accredited by the National Institute of Public Administration (NIPA). It was generally conducted over a period of 6–12 months. It had both theoretical and practical sessions which were supported by mentorship both during and after training. The training was delivered in four (4) workshops and the content arranged in six (6) modules. (Table 1)
The ZMLA course has been packaged in line with a recent study that recommended experimentation with action learning approaches, including a mix of formal training, on-the-job training, mentoring and support. So far, the ZMLA training has been offered to over 700 health workers in 27 districts between 2012–2014. The overall objective of the programme was to improve participants’ knowledge of management and leadership skills and help them employ these in their day-to-day duties. The first phase of the program was completed in September 2013; the second phase concluded a year later in September 2014. The third phase of 116 trainees completed the program in December 2014 (Table 2).
This paper presents results from the programme evaluation, which aimed to measure the extent to which the ZMLA training had contributed to improved management and leadership skills and working environment in line with the ZMLA conceptual framework, in addition to sharing lessons learnt through implementation of the course over a 4-year period.
Methodology
Study design
This was a cross-sectional mixed method study. Both qualitative and quantitative methods were used to collect information.
Target population
The study targeted health workers, stakeholders and course implementers. ZMLA trainees were targeted to gain perspectives on the extent to which the program affected levels of self-confidence resulting from knowledge gained. Perspectives were sought from both ZMLA and non-ZMLA trainees to measure changes in the work environment. Stakeholder perspectives were collected from trainers, and key informants involved in providing ZMLA training.
Eligibility criteria
All participants who took part in the training were eligible for the pre-post training interview. Those that had not completed both pre-post training questions were not included in the final analysis.
The workplace survey questionnaire was only administered to worksites of phase II. Phase I participants did not fill in the workplace survey at baseline.
Health workers found at the targeted workplace were eligible regardless of exposure to the ZMLA training. Key informant interview, included those working in their capacity as provincial, district and health facility managers regardless of the time they had been in the position. Those acting where interviews if the substantive respondent was absent during data collection.
Sampling
All participants for phases I and II were given a pre- and post-training questionnaire to assess changes in knowledge and skills. Eight (8) out of twenty-seven (27) districts were purposefully sampled to take part in the workplace survey. Qualitative data was collected from 70 key informant interviews (Table 2). The selection of respondents was purposeful and based on their positions within their organizations and their experience with the ZMLA program. Efforts were made to ensure that key informants from all stakeholders were represented. Each of the 10 provincial health offices was visited for key informant interviews.
Data collection
Routine data collection tools were used to monitor ZMLA progress and program quality. Knowledge quizzes were designed to test participants’ knowledge on several concepts before and after being trained. Participant feedback forms were collected after each training session. Participants were asked to develop group and individual case studies. The quality of all case study plans developed by participants was submitted to NIPA to assess participant’s ability to apply concepts learned.
Management and leadership survey
The management and leadership survey was used to measure changes in confidence and job motivation among ZMLA trainee respondents. The tool was administered twice to 164 phase I and to 280 phase II ZMLA trainees at the beginning and at the conclusion of each training. The self-perception confidence questions in the survey were premised on Albert Bandura’s concept of self-efficacy, which posits that a person who feels greater confidence in his or her ability to perform a specific behaviour is more likely to successfully perform the behaviour. The management and leadership tool used a 10-point Likert scale to measure responses to statements relating to management capacities, and a 5-point Likert scale to measure responses to statements relating to job-satisfaction and commitment to the organization. The questions were adapted from the Clinical Leadership Competency Framework (CLCF) and Medical Leadership Competency Framework (MLCF) of the United Kingdom. (www.leadershipacademy.nhs.uk). The tool was piloted and before being used in the current study. Before conducting the pilot, we first had consultative meetings with the Ministry of Health Technical Services team under which the Leadership development for MoH is hosted to review the tool. We also had consultative meetings with other stakeholder to contextualise and adapt questions to reflect the Zambian health care contexts. In addition, we reviewed the questions to ensure alignment with ZMLA course content which was the basis for the final evaluation. The draft tool was then administered to a sample of health workers in Lusaka who were not part of the ZMLA training. The responses were reviewed and adaptations were made based on pilot experience to come up with the final tool.
Workplace climate tool
The workplace survey questionnaire was only administered to worksites of phase II trainees that were trained from December 2013 to June 2014. Phase I participants did not fill in the workplace survey at baseline, making it inappropriate to use the data to measure perceived changes without relocation biases. Phase I data was thus not used in this evaluation.
The workplace climate survey was designed to measure existing conditions and changes in the workplace environment as scored before and after the course. The survey targeted everyone at a workplace, regardless of exposure to the ZMLA course.
The survey was self-administered with a ten-point Likert scale measuring personal perceptions of the work environment. The survey was administered before the start of the ZMLA trainings and shortly after the conclusion of the training (one to three months) in nine randomly selected districts health offices. Unlike the management and leadership survey, there was no pairing of respondents at baseline and end line evaluation and the data set had no individual identifiers to track individual responses at baseline and end line. The tool was adapted from the work climate review paper by Gershon,R et al 2004 [12].
Qualitative data collection
In-depth interviews with key informants were conducted in target provinces and districts. The selection of participants was purposeful. However, efforts were made to ensure representation from target districts and provinces. The main respondents were provincial medical officers, district medical officers, hospital administrators, ZMLA course implementers, mentors and selected stakeholders. Additionally, for each provincial office and selected district offices, 1–2 people trained in ZMLA were selected to take part in the study. We used an interview guide to collect the data. This was a flexible tool allowing for exploring issues as they arose during the interviews.
Data management
Quantitative data was entered into access database and exported to SPSS for analysis. Data was cleaned and all outliers were validated by checking against hard copy entries. Qualitative data was recorded using digital recorders. It was downloaded and backed up at the end of each day and later transcribed by trained research assistants.
Analysis of quantitative data
Quantitative data was exported to SPSS version 19 for analysis. Simple frequencies were used to explore the data. The basic unit of analysis was individual participants. Chi-square was used to compare between categorical variables and t-test to compare continuous variables. Statistical significance was set at 95% confidence level.
Qualitative data analysis
Trained research assistants experienced with qualitative methods transcribed data. After data cleaning, transcripts were exported into QSRNVivo 10 for analysis. Two of the authors (WM and AT) reviewed transcripts of interviews, validating pre-determined themes and identifying additional themes and subthemes that emerged. These themes where validated with other co-authors who confirmed the adequacy of the themes and subthemes. Two of the authors coded the data. In order to ensure consistency in data coding, we used Cohen Kappa, which is in-built into the Nvivo Software to check for inter-coder reliability[13]. This was found to be 0.72, corresponding to substantial agreement in coding by the two coders[13]. The pre-determined themes were based on the course contents and objectives (Table 1). Thematic analysis was then employed focusing on identifying patterned meaning across the data. We compared themes by districts, level of care, category of health workers and attendance of the ZMLA in order to understand the different perspectives of the ZMLA course.
Ethical consideration
The study obtained permission from the University of Zambia Biomedical Ethics Committee and the Ministry of Health. All participants who responded to the surveys were asked to provide written consent. All data were stored securely in a password protected Access database. All information was anonymised and no personal identifiers were used during analysis of data and publication.
Results
Demographic characteristics of participants
For key informant interviews, a total of 70 respondents were interviewed. Thirty (30) participants were from Provincial health office, twenty-four (24) participants were from the district teams, seven (7) and six (6) respectively were from the national level and health facility. Majority of the participants were medical doctors at both district and provincial level. (Table 3)
For the leadership and management questionnaire, 280 participants were interviewed in the phase 1 while 164 were interviewed from phase 2 giving a total of 444 participants. The majority were males (70%) and urban residents (59%) (Table 4).
Knowledge change pre and post workshop
On average, knowledge levels increased by 2.1 points (38%) after each workshop. The most significant change in terms of gain in knowledge was noted for workshops 1 and 2, by 3.0 (63%) and 1.9 (29%) points respectively. Workshops 4 showed the least change in skills score 1.3 (19%) points (Fig 2).
Case study performance by trainees
To supplement individual knowledge assessments through pre- and post-test quizzes, the programme required participants to demonstrate their understanding of various ZMLA concepts by applying them to practical case studies. The results indicated that participants performed best in the area of creating relevant and sound monitoring and evaluation frameworks (93%), but had difficulties in developing a model of care illustration (58%), delegation plan (56%), and job descriptions (50%) (Table 5).
Leadership and management skills
To understand how the training further improved individual leadership and management skills, both qualitative and quantitative methods were used. A comparison of the average self-rated scores from 444 management and leadership survey responses before ZMLA and after ZMLA training showed a significant increase in the proportion of participants that felt adequately trained to undertake management and leadership: Phase I from 63% before to 99% after, Phase II from 43% before to 98% after (Fig 3).
In addition to participants feeling adequately trained after undergoing the training, an assessment of self-perceived improvement in undertaking a range of management and leadership tasks, significantly improved. Fig 4 highlights self-perceived percentage changes in mean scores for six broad management and leadership tasks. The greatest changes were noted under the themes ‘problem solving, planning and managing programmes. The differences across all themes were statistically significant for all variables in the leadership and management tool.
Individual items change before and after ZMLA training
Individual item analysis for the 22 items on the leadership and management scores indicated that generally the baseline scores were high with average scores all above 7/10 for the first 22 items. The follow-up scores ranged between 8–9 points out of 10 for the 22 items. Result showed that there were improvements in leadership and management scores after the ZMLA training across all the 22 items. The differences were all statistically different for all the 22 items in the leadership and management tool. The highest mean score change was in the item relating to analyzing the model of care for health services to identify major bottlenecks. This was followed by the related item, which referred to the capacity to address gaps identified after problem analysis (Table 6). These findings validated results from the qualitative assessment, which showed that most participants appreciated the problem analysis so they could get to the root causes of problems (atomisation).
Qualitative findings complement the above quantitative results. In interviews, 55% (34/62) of the eligible respondents reported that they felt more prepared and motivated to perform their duties after their training:
“Work is a bit more enjoyable now because I know exactly what I need to do. So I have a reason to get up and go to the office because there is something that I have written down somewhere that this is what I need to do. Before it was very difficult for us to come up with work plans. It was like Greek, but now I know that I need to do this so that I can get that done, you plan your day well in advance because you know there’s something that you want. So I guess I’m more alive when I go to work now than going to work and thinking what are we going to do now.”
ZMLA trainee, Mbala
As an example, trainees felt confident in preparing their annual action plans. Trainees no longer copied and pasted plans from previous years but developed original action plans with competence and efficiency.
“…If you look at what we do, it’s mainly to do with planning. Like right now we are doing planning, we are developing our work plans, we look at the medium term expenditure framework going into the strategic plans…all those things that we were learning in ZMLA….,now we are not scared to go for planning…we now know how it should be done and my team is ready.”
ZMLA trainee, Nchelenge
Job motivation outcomes
To measure improvements in job motivation before and after undergoing ZMLA training, a series of questions were asked in the management and leadership survey. The focus of the assessment was: (i) commitment to career in the organization, (ii) optimism about future success in the organization, (iii) enjoying tackling new challenges arising in work, (iv) and feelings of positivity while working.
Fig 5 shows mean differences and associated confidence intervals for a 5-point Likert scale of before and after training. Commitment to a career in their organization and feelings of positivity when at work were the most notable changes in trainee job motivation. These findings were consistent across phase I and phase II of training.
Key informant interviews revealed that the ZMLA training contributed to their decision to stay in the public health service. The reasons given by majority of the trainees (52%) were that they now felt empowered and confident to be managers in the health sector and that they felt more motivated with skills and knowledge gained from the ZMLA.
“Job motivation has improved, of course may be not in terms of remuneration but in terms of motivation, because when you know what you are supposed to do it becomes a motivation on its own.”
ZMLA trainee, Mansa
Organizational benefits resulting from ZMLA trainings
Changes in the workplace climate
The characteristics of work climate respondents are presented in Table 7.
In evaluating the organizational benefits that resulted from ZMLA trainings, quantitative data from the workplace climate survey and qualitative themes derived from 70 in-depth interviews were analysed.
The overall workplace climate improvement after ZMLA was by 5.3%. The highest change was reported by non-ZMLA trainees with 8.6% improvement. Workplace climate change reported by ZMLA trainees was 3.8% improvement. These results suggest that non-ZMLA trainees might be a good source of information on workplace climate as they were independently confirming the effect of ZMLA training (Table 8).
Further analysis showed that respondents perceived improvements in the workplace environment especially in handling human resource management matters. The least improvement was noted in ethics and accountability. The calculated before and after percentage change for work environment themes ranged from 0.6%-13.4% (Fig 6)
Overall conduct of ZMLA and lessons learnt
Course duration
Participants from phase I of implementation were trained for a minimum of 12 months, which was seen as a very long time to complete the ZMLA course as explained by one respondent:
“I think our course was too long when you compare it to those who are doing it now. The course went on for over one year…it is very difficult for full-time workers to attend such a course…6 months is the ideal time for such a course.”
ZMLA trainee from phase I.
By the second phase of implementation, adjustments to the length of the course reduced the course to just six months. While this was seen as an improvement in the length of training, it was also associated with significant work overload for some of the trainees interviewed from the second phase of implementations. There was also a strong feeling that there was little time allocated for participants to go through practical examples and adequately share experiences. Many participants felt that adding another training day would make a big difference.
“In my view the course is overloaded…some of the concepts require more time. Some topics such as financial management need more time…but currently the time allocated is insufficient.”
Course implementer, Southern Province
“The time allocated for the course is too short. There are so many things to learn and some of the concepts are new to many participants…it would help even to add just an extra day.”
ZMLA trainee, Kalabo
Participant commitment
Participants’ commitment to ZMLA was not as high in the first phase as in the second and third phases. ZMLA was not well known, and it was unclear whether the training would be certified. After ZMLA was certified by NIPA and became better known, participants’ enthusiasm for and commitment to the course increased.
“In the initial stages when they didn’t know what the training entailed, there was a bit of resistance and some people couldn’t complete the course. Now everyone wants to get ZMLA training because they have discovered that you get the knowledge and a certificate that you can use elsewhere.”
Course implementer, Northern Province
This observation was reinforced by the reduction of attrition from as much as 32% in the first phase to just 8% in the second phase of implementation. The high dropout of phase I participants was also addressed in phase II by merging workshop and mentorship sessions into the same week rather than spacing sessions four weeks apart.
From all interviews conducted, it was apparent that ZMLA was popular among participants. One reason given across all cadres was that ZMLA was filling an important gap in leadership and management knowledge and skills among health workers. The course was said to increase self-confidence in leaders and provided them with practical tools to address everyday challenges.
“I think ZMLA is addressing what I would call the core problem from my perspective, from what we see in health sector, because the challenge really revolves around leadership and management. It comes down to the most efficient ways of using resources and someone has to have the leadership roles clearly outlined. I think it gives you the tools to break down the problem and see what solutions you can come up with at your level. This course boosts confidence to health workers”.
ZMLA Trainee and mentor
Human resource performance management
ZMLA training under the human resource module covered in detail the use and application of annual performance appraisal system (APAS). Evaluation findings showed that most managers started applying APAS in their workplaces after ZMLA training.
Before ZMLA training, APAS was viewed as a complex undertaking by virtually all Zambian health officials. After ZMLA training most participants were better equipped to complete the APAS than their non-ZMLA counterparts.
“We are now doing APAS, though not all staff are trained. With the few who were trained through ZMLA, we have been helping others to make individual work plans. There were a number of staff who were not confirmed but now some of them have been confirmed after completing their APAS.”
ZMLA trainee Chongwe
Many trainees who came from districts or provinces where the top managers were not ZMLA trained complained of delays in being appraised and wished their managers had undergone ZMLA training.
“What pains us is that, I have gotten this extra skill but nobody here is able to recognize it…I wish our boss had done ZMLA course, he will appreciate my skills much more.”
ZMLA trainee, Mansa
Change in meeting culture
Qualitative interviews showed improvements in the meeting culture and a greater appreciation for the importance of meetings. Almost all interviewees confirmed that the manner in which they now conducted meetings had improved after the ZMLA training. The workplace climate survey findings showed that the way meetings were conducted in the workplace was among the top three work environment themes that were most improved post-ZMLA training. Meetings were brief and to the point and included action items and responsible individuals.
“…..We have changed how we hold meetings. What we have started doing is that before a management meeting, we ask for departmental inputs …then set up an agenda with time allocation for each agenda item. If you want something discussed, it has to be sent early so that it is included in the agenda. We also circulate the agenda before each meeting. At the end of the meeting there are action points with responsible people clearly shown… this was not the case before ZMLA.”
ZMLA trainee, Chongwe
Shared vision, teamwork and coordination at workplaces
Interviewees reported that a shared vision was more emphasized following ZMLA training. Trainees had a greater understanding of their role and how it supported the shared vision. Improvements in management accountability and ability to delegate were also noted.
“…because most of the people am working with right now have also done the training, so when I begin to say ‘ok gentlemen this is what I think we need to do here’ they have an understanding of what exactly I am talking about so it’s easy for everyone to be on board. So because most of us have been trained at the district health office, it’s been smooth sailing so far.”
ZMLA trainee, Nyimba
“One of the things that has changed is [that there is] at least more coordination between the members at the DHO here and more importantly, being able to document whatever we discuss in management and having action points, and following those up. We have reached a point now where almost everyone knows about management and you can easily delegate.”
ZMLA trainee, Nchelenge
Improvement in shared vision, teamwork and coordination seemed to have improved more in work places where the overall manager was trained in ZMLA.
“Training of top leadership was extremely key. It was key because it also made sure that the leadership understands what’s happening. I think it puts the whole system of the health sector into perspective and I think it was really strategic to train PMOs and clinical care specialists ……indeed the people that may not have been trained in the concepts may not really appreciate them.”
ZMLA mentor and trainee, Solwezi
Improved financial management and accountability in workplaces
ZMLA training had improved financial literacy among trainees. Most trainees interviewed appreciated the role of the accountant and the need to adhere to financial regulations much more.
“The ZMLA training drew participants from all departments. The biggest challenge we had was managing the human resource, the people from different units. As accounts unit, I think we are very knowledgeable with financial regulations, but that was not the case with people from other departments. After the ZMLA training, we are seeing great improvement and compliance to financial regulations; there is now strict adherence to budgets. People here were not very comfortable with the bureaucracy we have in accounts, but after the training they appreciate and they understand the importance of observing these controls.”
ZMLA trainee, Kasama
Discussion
The study found that the ZMLA course improved individual knowledge and skills in leadership and management as demonstrated through individual score change before and after the course in various domains. This has implications for health system performance in the short and long term.
Overall, the assumptions of the ZMLA theoretical framework were supported by the results. The model suggested that through exposure to programme activities (workshops, mentoring, case studies), programme participants should demonstrate increased knowledge, skills, and confidence related to management and leadership functions, and experience improved job motivation, which should in turn affect their behaviour and health outcomes. There was evidence to suggest that exposure to the programme resulted in increased individual knowledge, skills and motivation, with most participants showing confidence in leadership and management practices. There was also a positive impact at workplaces with ZMLA trainees. There was some indication that service delivery also showed improvement, though this was not fully evaluated in this study. This shows that the ZMLA conceptual framework could be a useful model for analysing implementation processes and outcomes for courses aimed at improving leadership and management practices for health workers in low-income settings. This is an extremely important finding and provide opportunities for incremental knowledge gain over time especially in the context of health system strengthening where assessing of leadership and management remain poor despite the urgent need to improve how health systems are governed and managed to improve health outcomes.
Four characteristics distinguished the ZMLA course. These included the short course duration, team based training, mentorship and being a practical/hands on course. The ZMLA course was said to be different from other management courses in that it was done over a short period of time with less disruption to health services. The course was run for 6–12 months from the beginning to the end. Participants were away from their workplaces for a very short time, thus ensuring continuity of services. This was in contrast to the other courses which take up to 18 months with health workers being required to take full study leave, thus negatively affecting service delivery in districts. This course is therefore more acceptable and feasible in low-income settings were human resource challenges exist.
One key success factor to the ZMLA approach was team-focused training, where team members from the same district or health facility were invited for training at the same time and encouraged to work on problems related to their workplace in applying some of the principles learnt during the course. Literature has reported the benefit of team-based leadership training. Such an approach has been shown to leverage different strengths of team members and allow for team building, which is critical in organisation performance[14]. This was evident with trainees who participated in the ZMLA course, where respondents found team training beneficial in helping them to bond and discuss openly the challenges faced at their own institutions and coming up with interventions, which were linked to group case studies and was a requirement to complete the course. Strongly related to team training was the importance of having a team leader at the targeted institution taking part in the training. This was reported to be a key enabler in adopting and applying principles learned through the course. Places where the team leaders were left out of training struggled to implement what was learnt in leadership training due to resistance from top management who did not appreciate the new principles and approaches taught in ZMLA. Other studies have supported the need to include top leadership in such trainings [15].
It was obvious that the ZMLA course had made a difference for individual participants and in some places there was evidence to suggest that the work environment had changed. Some of the changes that were evident in the workplaces included the culture of holding meetings, application of Annual Perfomance Apprasail System (APAS) in the workplace, delegation of responsibility and more constructive feedback/better communication. Almost all respondents confirmed that the meetings were conducted meetings had improved after the ZMLA training. The meeting culture had changed with many more appreciating the importance of meetings. Meetings were now regularly scheduled and agenda items and minutes were sent in advance. One item that was dropped from almost all agendas as a result of exposure to ZMLA was “any other business”. This was found to be one of the reasons why many meetings were unpredictable and lasting for several hours. In terms of actual conduct of meetings, it was reported that now meetings were brief and to the point with clear action points and responsibilities assigned. This saved a lot of time, which in the past would be spent attending endless meetings. Similar findings have been reported in Gambia where both individual management skills and organisation performance improved following implementation of a similar course[16]. In health systems where health workers are expected to perform management duties and attend to patients, a course like ZMLA could be very important in ensuring better time management with better outcomes for the health system and patients.
The Ministry of Health recently introduced annual performance appraisal system (APAS) and many provinces and districts were oriented on the use of APAS before undergoing ZMLA training. Nonetheless, the application of APAS in the workplaces remained very low. Interestingly, most participants admitted that ZMLA had simplified the understanding of APAS, which many were struggling to master. Most participants who had done ZMLA were able to make individual work plans for appraisal and were also appraising those working under them. Application of APAS in the workplace was less evident if the top manager or team leader had not attended ZMLA training. Many participants who came from districts or provinces where the top managers were not ZMLA trained complained of delays in being appraised and wished their managers were also ZMLA trained.
Mentorship was a critical component of ZMLA training with all trainees being allocated mentors who would follow them up throughout the duration of the course, helping to think through challenges of applying lessons learnt and conducting case studies. Despite some few challenges, participants appreciated this approach. This is in line with what has recently been reported in literature that while didactic educational processes can be useful for providing ideas, material and motivation, it is not sufficient [1]. The extent to which leadership and management get integrated into the personal identity depends on the extent to which the skills and knowledge are integrated into daily practice. This should be reinforced through coaching and mentoring [1]. Mentorship formed a basis for trust and on-going support beyond the duration of the ZMLA course.
According to the ZMLA framework, the learning and application of ZMLA principles should translate into not only improved individual knowledge and work environment, but also lead to improved service delivery in terms of quality and coverage. We used selected case studies to evaluate the impact of ZMLA training on some services (Data not shown). Though there was some suggestion that some services improved following implementation of ZMLA case studies in some places, attribution of such effect to ZMLA remained a challenge. Firstly, the implementation of ZMLA case studies and principles was not done in isolation. There were many other activities which were going on in many sites and there were other partners who were targeting similar services. Other limitations in attribution were that the HMIS data was very poor in many sites. In addition, the time to observe changes in service delivery was too short. The challenge of linking leadership and management training to health outcomes has been reported in literature. Rigorous study designs that allow sufficient time to establish linkages between leadership training and health outcomes are being advocated [4, 8].
We explored the reasons why ZMLA was so popular and special among participants. One reason given across all cadre categories was that ZMLA was filling an important gap in leadership and management knowledge and skills among health workers. The course was said to increase self-confidence in leaders and provided them with practical tools to address everyday challenges. The course was specially tailored to the health sector and timed to minimise disruption to health services. The fact that the course was officially certified by the National Institute for Public Administration (NIPA), made it even more valuable for individual participants. They admitted that they could use not only the knowledge gained, but also the diploma to apply for promotion or other jobs. The popularity of the course among other health workers who had not undergone ZMLA training increased when they saw colleagues getting diplomas. Even those who had dropped out of the course in phase I came back to complete the course when they realised that they would get a diploma at the end of the ZMLA course. This finding emphasised the need for accreditation of such courses especially if they are targeted at busy health workers who may not be willing to attend non-recognised courses.
While the ZMLA course had many positive attributes, it also had challenges. Some of these were improved upon in the second and third phases while other challenges still remained. Generally, participants agreed that the timing for the ZMLA course was too short. Many participants struggled to work through group and individual case studies, while others felt some components of the theory were rushed and there was little time for group work and practical sessions. One component of the course that was poorly organised was mentorship after completing the course work.
Interestingly, the graduation of participants was only based on course attendance and writing a case study proposal, but not implementation of the case study. This contributed to failure to implement case studies by many ZMLA trainees. Nonetheless, it was felt by many respondents that implementation of case studies was crucial in ensuring trainees actually translated the theory into everyday practice.
This study looked at ways in which the ZMLA course could be made sustainable. It was clear from the responses that most participants wanted ZMLA to continue in its current form and even extended to other cadres at health facility and community levels. Unfortunately, the course was marketed as a free donor-supported programme. In addition, most health workers expected to be paid allowances for attending the course. Interestingly, most respondents admitted that the way the course was run was expensive and was not sustainable. It was suggested that if the course were institutionalised by the University of Zambia or similar institution, this would make it cheaper and more sustainable.
The study had several limitations: The study used purposeful sampling to identify respondents and qualitative methods were used to collect information from key informants. This limits the generalizability of our findings as the sample might not be representative. We triangulated data sources to improve validity of the results. The self-reporting responses which were used in the pre-test and post-tests quizzes, workplace climate and leadership and management tools might be affected by a tendency to report desired results and selection of higher scores. This was probable as noted in the scores, which tended to be generally high pointing to the ceiling effect in the instrument we used. However, it was explained to the respondents that they were to be honest with answers given. The study design did not include a comparison group and there was no counterfactual to compare the effect of ZMLA. It was therefore not possible to attribute the effect on performance, such as change in service delivery to ZMLA as there were other confounders that could not be controlled for.
Conclusion
The study showed that the ZMLA course was well appreciated by individual trainees and stakeholders. The course was seen by many as a motivator, a source of skills and knowledge, and a tool that empowered health system managers to understand their roles and challenges. Evidently, the course had improved leadership and management knowledge and skills for targeted public healthcare workers. There was evidence suggesting that workplace climate had improved in sites with ZMLA trainees.
Acknowledgments
The authors would like to acknowledge the Ministry of Health (MoH) and the Ministry of Community Development, Mother and Child Health (MCDMCH) for making this evaluation possible. We sincerely thank the provincial and district health teams for their cooperation and feedback during the data collection process. We would also like to thank all health professionals who participated in the study and all Management Specialists under the Zambia Integrated Systems Strengthening Program (ZISSP) for the logistical support rendered to the evaluation team.
We like to thank ZISSP for the financial and material support rendered to the evaluation team.
Author Contributions
- Conceptualization: WM ATVM AK RM KC DM.
- Data curation: WM ATVM AK RM KC DM.
- Formal analysis: WM ATVM AK KC.
- Investigation: WM ATVM AK RM KC DM.
- Methodology: WM ATVM AK RM KC DM.
- Project administration: WM ATVM AK RM KC.
- Resources: KC RM.
- Software: WM ATVM AK.
- Supervision: WM ATVM AK RM KC DM.
- Validation: WM ATVM AK RM KC DM.
- Visualization: WM ATVM AK RM KC DM.
- Writing – original draft: WM ATVM AK RM KC DM.
- Writing – review & editing: WM ATVM AK RM KC DM.
References
- 1.
Judith Daire, Gilson L, Cleary S. Developing leadership and managementcompetencies in low and middle-income country health systems: a review of the literature. 2014.
- 2. Deber RB. Thinking about accountability. Healthcare policy = Politiques de sante. 2014;10(Spec issue):12–24. pmid:25305385.
- 3. Chigudu S, Jasseh M, d’Alessandro U, Corrah T, Demba A, Balen J. The role of leadership in people-centred health systems: a sub-national study in The Gambia. Health policy and planning. 2014.
- 4. Ciccone DK, Vian T, Maurer L, Bradley EH. Linking governance mechanisms to health outcomes: a review of the literature in low- and middle-income countries. Social science & medicine. 2014;117:86–95. pmid:25054281.
- 5. Seims LR, Alegre JC, Murei L, Bragar J, Thatte N, Kibunga P, et al. Strengthening management and leadership practices to increase health-service delivery in Kenya: an evidence-based approach. Human resources for health. 2012;10(1):25. pmid:22931468.
- 6. Daire J, Gilson L. Does identity shape leadership and management practice? Experiences of PHC facility managers in Cape Town, South Africa. Health Policy Plan. 2014;29 Suppl 2:ii82–97. pmid:25274644.
- 7. Gilson L, Schneider H. Commentary: Managing scaling up: what are the key issues? Health Policy Plan. 2010;25(2):97–8. pmid:20053734.
- 8. Hastings SE, Mallinson S, Armitage GD, Jackson K, Suter E. Mind the gap: governance mechanisms and health workforce outcomes. Healthcare policy = Politiques de sante. 2014;10(1):e108–14. pmid:25410700.
- 9.
Kirkpatrick D, Kirkpatrick J. Evaluating Training Programs: Four Levels: Berrett-Koehler; 2006.
- 10. McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health Education. 1988;15: 351–77.
- 11.
LeMay N, Ellis A. Evaluating leadership development and organizational performance. Management Sciences for Health, 2006.
- 12. Gershon RR, Stone PW, Bakken S, Larson E. Measurement of organizational culture and climate in healthcare. J Nurs Adm. 2004;34(1):33–40. pmid:14737033.
- 13. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276–82. pmid:23092060.
- 14. Salas E, Almeida SA, Salisbury M, King H, Lazzara EH, Lyons R, et al. What are the critical success factors for team training in health care? Jt Comm J Qual Patient Saf. 2009;35(8):398–405. Epub 2009/09/02. pmid:19719075.
- 15. Aarons GA, Sommerfeld DH. Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423–31. Epub 2012/03/28. pmid:22449648.
- 16. Conn CP, Jenkins P, Touray SO. Strengthening health management: experience of district teams in The Gambia. Health policy and planning. 1996;11(1):64–71. pmid:10155879