Skip to main content
  • Research article
  • Open access
  • Published:

To what extent can the activities of the South Australian Health in All Policies initiative be linked to population health outcomes using a program theory-based evaluation?

Abstract

Background

This paper reports on a five-year study using a theory-based program logic evaluation, and supporting survey and interview data to examine the extent to which the activites of the South Australian Health in All Policies initiative can be linked to population health outcomes.

Methods

Mixed-methods data were collected between 2012 and 2016 in South Australia (144 semi-structured key informant interviews; two electronic surveys of public servants in 2013 (n = 435) and 2015 (n = 483); analysis of state government policy documents; and construction of a program logic model to shape assessment of the feasibility of attribution to population health outcomes).

Results

Multiple actions on social determinants of health in a range of state government sectors were reported and most could be linked through a program logic model to making some contribution to future population health outcomes. Context strongly influences implementation; not all initiatives will be successful and experimentation is vital. Successful initiatives included HiAP influencing the urban planning department to be more concerned with the health impacts of planning decisions, and encouraging the environment department to be concerned with the health impacts of its work.

Conclusions

The theory-based program logic suggests that SA HiAP facilitated improved population health through working with multiple government departments. Public servants came to appreciate how their sectors impact on health. Program logic is a mechanism to evaluate complex public health interventions in a way that takes account of political and economic contexts. SA HiAP was mainly successful in avoiding lifestyle drift in strategy. The initiative encouraged a range of state government departments to tackle conditions of daily living. The broader underpinning factors dictating the distribution of power, money and resources were not addressed by HiAP. This reflects HiAP’s use of a consensus model which was driven by (rather than drove) state priorities and sought ‘win-win’ strategies.

Peer Review reports

Background

There have been many international calls for effective action on the social determinants of health (SDH) involving a range of government sectors [1,2,3,4,5,6]. Health in All Policies (HiAP) was developed by the European Union, and the WHO defined it as: ‘an approach to public policies across sectors that systematically takes into account the health implications of decisions, seeks synergies, and avoids harmful health impacts in order to improve population health and health equity’ [7]. HiAP initiatives have been adopted in many cities, regions and countries (see for example [1,2,3]). Establishing the health impact of HiAP is methodologically challenging. There are multiple problems with applying randomised control trials to complex entities such as cities, regional and national governments [4, 5, 8]. Foremost among the problems are the difficulties of establishing control units that are really identical, the complications of ‘controlling’ the activities of experimental or control administrative units, then making realistic claims about attribution of observed differences in health status in complex systems where many factors affect health and outcomes may take years or decades to become apparent. In response, evaluators have advocated the use of logic models based on chains of causality to assess impact [6, 9, 10]. Such models are particularly useful when combined with insights from the critical realist evaluation approaches. A central aspect of this form of evaluation is a concern with causality and the identification of causal mechanisms in social phenomena like a HiAP initative in a manner quite unlike the positivist search for causal generalizations [11]. Emphasis is placed on the contribution that a policy intervention makes to an outcome and views this contribution within its social, economic and political context [12]. These models map context, detail mechanisms that influence pathways to outcomes, and measure outputs and outcomes. Thus logic models using a critical realist approach hold most promise for untangling the question of whether HiAP is effective in promoting health [13, 14]. Our study is the first application of such models to HiAP to assess whether it has contributed to health.

The context of South Australian HiAP

In 2007 the South Australian (SA) Government adopted HiAP, building on a long history of healthy public policy advocacy and innovation in South Australia [15]. By 2008 a dedicated unit had been established within the Health Department, and HiAP was established as a cross-government process formally endorsed by Cabinet. Governance for the initiative was linked to the State Strategic Plan, and an intersectoral ‘Health Lens Analysis’ process was implemented [16]. HiAP has operated continually since this time. SA HiAP involves multiple partners with sometimes divergent agendas implemented in, and influenced by, changing political, organisational and economic contexts, which are vital to understanding its implementation and impact [17].

South Australia’s population is 1.6 million with 1.2 million in its capital, Adelaide. From 2014 to 2016, South Australia has had a high life expectancy of 80.4 years for males and 84.5 years for females, increasing by 1.8 years (males) and 0.9 years (females) over 10 years [18]. There are, however, significant health inequities. In the period 2009–2011 Aboriginal and Torres Strait Islanders died 10 years before other Australians, and people in the lowest socioeconomic areas lived about 3 fewer years than those in the highest areas [19]. Over the study period (2012–2016) there were significant changes in the economic fortunes of the state that affected HiAP. Less favourable economic circumstances meant social priorities were subsumed by an economic focus leading to cost cutting in the public service. The HiAP initiative survived by aligning with the new mandates and ensuring it retained relevance to other sectors [20].

Implementation of HiAP used limited resources, including a small and varying number of staff (6 FTE at full complement). Staff costs in 2016 were approximately $550,000 pa. The SA Health Department budget for 2015–16 was $5.8 billion and HiAP totalled 0.00948% of that budget.

South Australia’s authorising environment at the time gave HiAP a mandate to work with other sectors on government priorities. The authorising environment included the government’s requirement for intersectoral collaboration to achieve the targets in South Australia’s Strategic Plan, and reporting requirements to Cabinet to ensure departmental accountability for achievement of these targets (discussed further below). While engaging with HiAP, other sectors conducted their normal business in an adapted way and some provided in-kind support and contributed limited additional resources.

This overview paper reports a five-year study using a theory-based program logic framework and addresses the question: To what extent can the activities of the South Australian Health in All Policies initiative be linked to population health outcomes using a program theory-based evaluation?

Methods

We collected mixed-methods data (described below) within a theory-based program logic framework [12] over five years (2012–2016). This paper reports on the population health outcomes that, using the theory-based program logic framework, can be claimed to be likely from the SA Health in All Policies intiative. It draws on new data analysis as well as prior data analyses from the evaluation that have been reported in other papers [20,21,22,23]. The other papers consider HiAP processes and show how we have developed and applied the theory-based logic model to understand HiAP’s influence on population health outcomes. These papers and their key findings are summarised in Table 1. Drawing on these prior papers in addition to new analyses undertaken for the current paper is vital because the evaluation has produced considerable amounts of data that cannot be adequately presented and discussed within the contraints of a single overview paper.

Table 1 Summary of key findings from published papers

The methods are summarised in Table 2 and elaborated on below and have been described previously [20].

Table 2 Methods

Development of a program logic model

At the outset of the research, a collaborative process was used to develop a theory-based program logic model (PLM) as part of our theory-based evaluation framework from which the assessment of health impact could be made (for details of the framework and process of developing it see [8, 12]). The model in Fig. 1 outlines the program theory underpinning HiAP and applies contribution analysis to determine the extent to which the activities and strategies of HiAP can be argued to have contributed to improved health. The strength of evidence we have to support our claims in relation to the potential for SA HiAP to achieve health outcomes in each part of the model in Fig. 1 is indicated with green shading equating with strong evidence, orange indicating moderate, and red indicating evidence relying on a projected contribution, based on the body of existing evidence indicating what health outcomes are likely to flow from particular actions.

Fig. 1
figure 1

South Australian Health in All Policies initiative program logic

Semi-structured interviews with public servants

Between January 2013 and June 2016 144 semi-structured interviews were undertaken. 53 of the interviews were with staff from the SA Health Department, and 51 involved staff from 15 other departments/agencies of the SA State Government who had been involved with the HiAP approach between 2007 and 2016. The 15 other departments/agencies included: community services, education, justice, transport, governance, infrastructure, employment, trade, and natural resources. Five of the interviews involved academics who had knowledge and experience of the HiAP initiative and four involved politicians or political staff (Table 2).

The evaluation was informed by the application of social and political science theory so that interview questions considered, for example, agenda setting, the role of actors as champions in diffusing HiAP ideas, and why and how particular HiAP activities and outputs lead to distal health outcomes (further details are available at [12]). The later interview schedules were adapted in light of emerging findings from the earlier interviews and the survey, so that for instance in the later interviews the impact of the worsening economic situation in South Australia on the public service was included.

Six of the research team, all very experienced in qualitative interviewing, conducted the interviews using the pre-prepared interview schedules which, in relation to outcomes, sought interviewee’s views about key markers of success of the SA HiAP approach, the extent to which understandings about health and wellbeing have been incorporated into participants’ thinking and other departments’ core business, and the extent to which South Australia benefited from its involvement with HiAP. The interviews were recorded and transcribed verbatim for all but two respondents. The interviews averaged 38 min, ranging from 10 min to 1 h 35 min. Interviewees were able to review their transcript and seven chose to check and amend their transcript by either editing the transcript directly or sending email notes to clarify or elaborate on a particular aspect of the interview. Two people requested written notes be taken by the interviewer rather than a recording. In these cases the notes were checked by the interviewee.

Electronic survey of public servants

An online survey of the HiAP policy network in the SA public sector was conducted in 2013 and repeated in 2015. HiAP staff assisted the research team to identify public servants who had had some contact with HiAP since 2007. In 2013 and 2015 the network involved 435 and 483 public servants respectively. The survey samples for 2013 and 2015 were selected from the identified public servants and included only people who were working within the SA Government at the time of each survey. In 2013, 373 public servants were invited via email to participate in the first survey. 168 (45%) of these people provided meaningful responses by answering survey questions beyond the initial demographic questions. Six (2%) people answered only the demographic questions and their responses were excluded from the analysis. 199 (53%) people did not respond at all. In 2015, 339 people were invited via email to participate in the second survey. 151 (45%) of these people provided meaningful responses by answering survey questions beyond the initial demographic questions. 25 (7%) people answered only the demographic questions and their responses were excluded from the analysis. 163 (48%) people did not respond at all. Each potential respondent was contacted four times [24]. The surveys sought information about respondents’ awareness of the HiAP approach, their experiences of collaborating with HiAP, and their perceptions of the outcomes of HiAP work. The survey data are used in this paper to report on the perceptions of public servants on the impact of HiAP, on their understanding of social determinants, and the impact that the work of their department has on population health.

Case studies

The research included case studies of the major work conducted by the HiAP team either as Health Lens Analysis projects or subsequently as part of other government initiatives (further details of these case studies are available at [21]). These case studies have been used to frame the case for the contribution analysis and attribution to projected population health outcomes.

The criteria we used to select the Health Lens Analysis (HLA) projects (discussed in more detail below) as case studies were: a) had progressed sufficiently to allow assessment of what outputs had been produced; b) involved a number of different departments to ensure collection of a range of sectoral views; and c) incorporated a focus on Aboriginal and Torres Strait Islanders as well as other population groups. During selection we ensured that we did not select HLA projects that were led by departments that were undergoing severe cuts or restructures. Securing interviews in such departments would have been logistically and politically difficult. Furthermore, we selected some case studies that followed the usual HLA process, and others in which a modified process had been applied to manage decentralised decision making processes and/or alternative managerial structures. Case studies were selected progressively as the research project continued. This allowed us to select some examples of HLAs that had been initiated to meet South Australia’s Strategic Plan targets, as well as some that had been initiated to fulfil the requirements of the newer legislative driver, the SA Public Health Act.

While there may have been potential for the findings from the multiple datasets to be conflicting, we found that rather they built upon each other and were used iteratively in our analysis. Semi-structured interviews were undertaken progressively throughout the research – before, during and after the collection of data from other methods. Follow up interviews were undertaken with key individuals to understand the changing contextual factors that were identified during the surveys, as well as the developments in particular case studies that were identified during the document analysis. Interview data also informed the development of survey questions and guided examination of case study documents. The surveys provided a greater breadth of data than the interviews, which involved deep examination of particular informants’ experiences. As a result of this research approach, each method supported and informed the other.

Analysis of data

A collaborative thematic analysis of the interview transcripts was conducted using the qualitative analysis software NVivo 11. The initial round of open coding involved two members of the team reading the transcripts to identify central themes followed by collaborative coding sessions with other investigators. Selective coding was applied to examine respondents’ discussion of key outputs and outcomes. During the selective coding process, we used the program logic model [12] to identify and organise the data against the three most relevant components of the model that link to the anticipated program outcomes: outputs, impacts on the policy environment, and policy processes and settings supportive of health, wellbeing and equity. Numerical data from the survey were analysed using SPSS software. Cross tabulations compared relationships between responses to selected questions.

Throughout the research, emerging findings were fed back to participants and informed the ongoing implementation of HiAP. For the duration of the research, the program logic framework provided a means to track changes to HiAP and the program theory inherent within it [12].

All data collection activities received prior approval from the Flinders University Social and Behavioural Research Ethics Committee and the SA Health Human Research Ethics Committee. Informed consent was provided by all participants in the research prior to their interview.

Results

Our findings are shaped by the program logic model (see Fig. 1) and describe how HiAP encouraged action on social determinants, and the likelihood that these actions have led or will lead to improved health and equity.

Action on the social determinants of health

The program theory held that improved health would result from intersectoral policy development to encourage action on the SDH. From 2009 to 2013 HiAP used Health Lens Analysis (HLA) to work with other sectors to determine what action on SDH was required [25]. HLA is a collaborative process between health and another sector to examine the health impacts of a given policy area [8]. The HLAs were effective in highlighting relevant SDH and were broadly supported by policy actors from the other sectors [20].

Table 3 presents an analysis of the program logic strategies relating to the main HiAP activities in Fig. 1:

  • Utilisation of governance systems to support SDH and equity

  • Development of relational systems to support action on SDH

  • Joint problem identification and decision making between HiAP and other sectors

Table 3 Reflection of aspects of program logic in SA HiAP action on the social determinants of health (2007–2016)

Table 3 shows immediate and intermediate outcomes for each area and the longer-term likelihood of health outcomes.

Utilisation of governance systems to support SDH and equity

South Australia’s Strategic Plan (SASP) provided an initial authorising environment for HiAP. It included targets related to the SDH which departmental chief executives were accountable for achieving. This increased the chief executives’ motivation to co-operate with HiAP and provided a clear commitment to improving health and wellbeing.

In 2011, the SASP was supplemented by seven strategic priorities and HiAP led an HLA to determine how they related to health, for example exploring the relationships between the strategic priority ‘growing advanced manufacturing’ and population health and wellbeing, including through economic development, employment opportunities, and development of the advanced health-manufacturing sector. The health logic was that increased employment and economic development would have a positive health impact. In 2016 the government committed to making South Australia A State of Wellbeing and HiAP was central to the development of an intersectoral wellbeing statement which incorporated some SDH elements.

A new Public Health Act reflecting new public health principles provided an additional authorising environment for HiAP and the legislative mandate for two important developments: Public Health Partner Authorities (PHPA) and Regional Public Health Plans (see Table 3 for examples).

Development of relational systems to support action on SDH

Culture change in the SA public service

HiAP achieved some success in bringing about culture change in the public service whereby public servants in a wide range of departments came to appreciate that their work had an impact on population health and wellbeing, and that maximising the positive health impact was part of their role. Data from the qualitative survey and interviews with public servants suggest HiAP increased public servants’ awareness of the health impacts of their agencies’ policies. In the 2015 survey 55.4% of respondents reported that they could see clear links between the work of their department and the SDH, with 53.2% of respondents either agreeing or strongly agreeing that collaborating with HiAP had increased their understanding of the SDH and equity. The interviews indicated that HiAP had built networks, created synergies and broken down silos in the public service. Executives from transport, urban planning, education and infrastructure stressed that health was now routinely considered in their planning processes. For urban planning staff, this meant that the health staff were invited to comment on their revised planning legislation and health considerations were built into both the legislation and plans. Planning staff also reported “heightened recognition” of the importance of neighbourhood design to health, and health was considered in the planning of transit-oriented developments more than before HiAP’s involvement. Education staff reported that the health impact of their core work was more widely appreciated. Strength of attribution to long term outcomes can be seen in the traffic light shading in Fig. 1.

Public servants also noted that HiAP initiatives were “on a convergent path” with the agendas of their departments. A health public servant noted that a department “might have been thinking about it [a health focus] or working on it, considering it”, and HiAP had “given a leverage point to make it across the line”. An example was where the urban planning goal of creating a ‘vibrant’ city converged with the health goal of encouraging cycling for health benefits as cycling makes the city more ‘alive’. The Department of the Premier and Cabinet saw HiAP as one of the tools that produced coherent policy.

Public servants noted that when they changed work roles they continued to apply knowledge gained through involvement in HiAP. One survey respondent noted that in their new role “the understandings and benefits of this approach remain with me”. A departmental head made a similar comment about using her new position to embed health thinking.

Joint problem identification and decision making between HiAP and other sectors

Initiatives in this section are concerned with intersectoral work to address an area of government activity which affects SDH. The HiAP team sought to identify and negotiate HLA projects within the core business of the partner agency that were supported by research evidence for potential positive health outcomes.

Employment and education

Four HiAP projects directly addressed the two vital SDH of employment and education. A program to encourage parental engagement in children’s literacy was trialled at four schools. Public servants from Education in part attributed to HiAP the adoption of a focus on parental engagement in the revised Numeracy and Literacy (birth to 18) Strategy [26]. The well-established link between literacy and health supports a conclusion that increased literacy skills as children grow up would improve health [27, 28].

One HLA concerned the health and wellbeing of international students. Higher education is an important part of the SA economy. A report into overseas students’ experience in Adelaide had concluded that students had very little understanding of the Australian health system [29]. Our analysis indicated that while significant issues concerning SDH for international students were raised, the output from the project was a guide to the Australian health system for these students. The department responsible for higher education reported finding the HLA process overly complicated and prolonged, and a senior executive from that sector commented that “the reality is that we could do it on our own anyway and we could actually probably do it more efficiently and see those returns and the benefits actually a lot quicker”. In this case HiAP started with a more ambitious project concerning sexual health issues and safety for overseas students but the process of collaboration whittled the focus of the project down to a much less ambitious information pamphlet.

In terms of employment, HiAP’s involvement in planning for the development of a regional area of SA that was expecting a mining boom encouraged public servants to focus on the health and wellbeing aspects of the plan by producing a social health atlas of the region. Although the mining boom did not eventuate, public servants from the industry sector reported that HiAP involvement slightly increased awareness of SDH and laid the basis for future joint work which has eventuated. Another employment-focussed project with a different sector did not proceed, and our analysis indicates this was because the department concerned was beset with budget cuts and staff losses, and previous involvement with HiAP was perceived to have few benefits. Overall given the economic headwinds faced by SA it would have been hard for HiAP to make much contribution in an increasingly adverse employment environment in which casualization was increasing and unemployment rising [30].

Sustainable urban and regional planning and infrastructures

Some HiAP HLAs were concerned with planning and sustainability (see Table 3). Each showed explicit causal pathways between HiAP processes and likely future health outcomes. HiAP contributed to the state’s aim of making the capital Adelaide less carbon dependent, including increasing active forms of transport. Initially this HiAP contribution was through three HLAs on transit-oriented developments. The link to health (summarised in [31, 32]) is that a more carbon efficient economy would contribute to a sustainable environment, and active transport options would mean more people exercise and benefit from casual social interactions. Our study indicates that HiAP played an important role linking environmental sustainability directly to health outcomes supporting policy coherence. For instance an executive in the industry sector spoke of introducing health indicators into consideration of sustainable mining development to develop healthy communities.

The relational aspects of HiAP were reinforced following implementation of the Public Health Act which provides a legislative mandate for intersectoral action. The Act established Public Health Partner Authorities (PHPA) whereby agencies could sign a formal agreement with the Department of Health concerning action on social determinants. One example was the PHPA agreement between health and the environment sector on Healthy Parks, Healthy People. This agreement provided a legislated institutional framework for co-benefits to be achieved by health and the environment sectors including helping to secure longer term population health and wellbeing outcomes as a result of increased green space in which people can spend time [33, 34]. This initiative is a whole-of-population initiative with a special focus on Aboriginal health, and has the potential to reduce inequities over the longer term. The strength of attribution can be seen in the traffic light shading in Fig. 1. The PHPA agreements provide instrumental support for the relational work of HiAP and made a direct impact on the policy environment by supporting converging agendas, strengthened alliances and organisational capacity for intersectoral work. Finally, our analysis showed that many of the 2007 HiAP principles were reflected in SA local government’s Regional Public Health Plans. These plans partly reported existing actions by councils to address the SDH and their intentions to expand their actions.

Healthy weight

The SASP set the goal of increasing the percentage of the state’s population at a healthy weight. The need for such a goal is evident – in 2014–15, 65.8% of South Australians over 18 were either overweight or obese, increasing from 50% in 2004–05 [35]. HiAP commissioned work to determine what government departments were doing and could do to contribute to achieving the goal. Many actions were identified including creating community vegetable gardens in public housing areas, intensifying active transport initiatives, increasing visits to parks, and increasing healthy food supply in prisons [36].

Aboriginal drivers’ licensing

Aboriginal drivers’ licensing was a multi-sectoral project with initially wide-ranging aims that were finally reduced to interventions in one remote Aboriginal community in the state. As a direct result of HiAP, legislative and policy changes were made to make the licensing system fairer for Aboriginal people living in this community. HiAP was the first of multiple initiatives addressing this issue, making attribution difficult. However, HiAP evidence gathering and recommendations did influence the work being undertaken. The eventual changes increased driver training for some Aboriginal people, so that road injury and deaths can be expected to be avoided in the long term and fewer Aboriginal people are likely to be imprisoned for licence infringements.

Co-benefits approach

The selection of HLA projects was based on a co-benefits approach, requiring the mutually agreed identification of projects that would address the objectives of the participating partner agency while having potential health outcomes according to HiAP’s research evidence. While the focus of the topics negotiated between HiAP and their partner agencies was clearly the core business of the partner agencies (eg a child literacy project with the Education Department, Aboriginal road safety with the Transport Department, and increasing green space with the Environment Department), the involvement of HiAP ensured a new and more critical focus on the health implications of policy decisions in partner agencies’ consideration of their core business. This enabled other sectors to develop understanding of their role in addressing the social determinants of health, resulting in increasing coherence across government policies and an increased awareness among participants of the broader health and wellbeing implications of their agencies’ policies for the SA community.

Discussion

Our main finding is that, through using a pragmatic and theory-based program logic approach to causal analysis, SA HiAP can be judged to have made a modest contribution to actions likely to have improved population health in South Australia. It did this using the activities and policy processes shown in Fig. 1. The intiative was small: the SA Health budget for 2015–16 was $5.8 billion. HiAP totalled 0.00948% of that budget. Judged against this modest input, impact in terms of changing the culture of the public service and encouraging health to be understood as an important consideration in other policy areas, especially in urban planning and the environment, was significant. We argue that most actions listed in Table 3 are likely to lead to longer term health benefits given that the evidence links these interventions to health outcomes. Of course whether those health outcomes eventuate will also depend on the South Australian and global context. Contextual factors reducing the support from other departments accounted for one initiative not proceeding. In another instance we observed lifestyle drift from an initially wide-ranging identification of social wellbeing concerns for overseas students to an information pamphlet.

The activities and impacts shown in green in Fig. 1 were found to have occurred and collectively to have built an authorising environment for a range of policy processes that support health and wellbeing. Our evidence indicates that HiAP helped public servants to appreciate how their sectors affected health [20]. We predict this will be positive for the state’s future but also caution that the government’s response to the economic climate by reducing the public service results in staff turnover, loss of capacity and knowledge. This may have a negative impact, outweighing the positive HiAP effect. Elsewhere we have noted that the pursuit of health equity was more prominent in the early days of HiAP where some rhetorical support existed for progressing an equity agenda [21]. Subsequently, however, economic pressures resulted in the government narrowing its priorities to economic goals, and reflecting HiAP’s dependency on its political context, little was done to address inequities in our study period.

A major strength of our study is that it is the only application of a theory-based program logic to assess the likely contribution of a complex, longer term public health initiative. While we can only make limited attributions of causality of HiAP to improved health and wellbeing in South Australia, we can do this within a predictive model which sets the likely outcomes and the processes by which they are achieved [12]. The use of program logic to support a narrative argument concerning the likelihood of long term health impact is novel and has not been applied longitudinally to HiAP in any other setting. This study demonstrated that producing the evidence for HiAP’s impact called for in Leppo et al. [37] is possible with a longitudinal design and partnership between public servants and academics. Further, Guglielmin et al. [38] conducted a scoping review of local implementation of HiAP and concluded that contribution to theory development on HiAP implementation requires further research that ‘specifically investigates the facilitators and barriers of HiAP locally within their political and policy context’. This is precisely what our research has done by highlighting the ways in which the South Australian economic and political contexts had a significant impact on implementation and so influenced the likelihood of health outcomes.

Limitations to our research are that the attribution of causality is necessarily tentative and relies on imputed evidence. In the actions we observed, HiAP was a facilitator of change with many partners. This makes determining its contribution difficult. While the program logic shows a smooth flow through of action emphasising the contribution of the HiAP initiative, the actual picture is messier, with contributions, and in some cases the major contribution, coming from other sectors. However, interview respondents consistently informed us that in every case HiAP played a facilitating role and we were able to document that over time. There are also limits to how much account we could take of context. Labonté [39] noted in a commentary responding to our theory-based program logic that our work did not include ‘considerations of political and economic shifts at the national or global scales (eg, hyperglobalization, capital mobility, economic financialization, and tax competition)’ and that it ‘assumes that the policy actors at the South Australian state level have the capacities to implement HiAP in ways that can institutionalize sustained improvements in health equity’. Our program logic model was able to take into account some globalised context and did this, for example, in terms of the closure of an automotive manufacturing plant and the general impact of de-industrialisation in South Australia. It is, however, difficult to measure and account for the kind of global changes Labonté refers to in research which focuses on local activity. Institutional and political constraints meant SA HiAP focused on improving daily living conditions and was not able to address the underlying drivers of health inequities.

Conclusions

We draw five key lessons from our evaluation of SA HiAP to assist other jurisdictions using or considering using the approach. These are:

  1. 1.

    Context is vital in shaping implementation. It both helped and hindered HiAP in a range of ways. Constant monitoring of the context was important to the implementation and evaluation of HiAP. Such careful attention helps the initiative to adapt to changing political and economic circumstances and helps evaluators make sense of why HiAP was adapting and how successfully it did so.

  2. 2.

    HiAP focused mainly on the implementation of initiatives that would change daily living conditions and did not challenge the basic structures of society which determine inequities. None of its actions were concerned with the SDH defined as changing the underlying distribution of power, money and resources [40]. This reflects several factors. SA HiAP was implemented at a level of government which does not control many aspects of the distribution of resources. It was also a project built on achieving a high degree of consensus (win-win strategies were favoured) and had very few avenues for citizen participation. Redistributive policies are likely to be contentious and require strong citizen involvement [21]. They will also require strong political leadership and policy development which includes advocacy for the health and social benefits of a more equal society. HiAP’s emphasis on consensus and win-win solutions may not always be appropriate when a strong advocacy approach is required, and so other approaches to intersectoral action for healthy public policy will still be required.

  3. 3.

    Regional government-based HiAP initiatives need to be complemented by those in national governments where the levers to change the distribution of power, money and resources are more powerful.

  4. 4.

    Not all the initiatives HiAP embarked on met with success. This is not surprising when the initiatives were dealing with many sectors with varied agendas in a context in which the public service was being subjected to considerable cutbacks. HiAP is an innovative approach at the cutting edge of the new public health and has to be allowed to experiment and sometimes fail.

  5. 5.

    HiAP was, in the main, successful at keeping the focus of projects on whole populations and universal policies and initiatives which then require relatively small shifts to have a significant impact across a population [41]. We found evidence of reversion to a lifestyle focus but this was very minor compared to the initiatives’ success in focussing on population-wide strategies, which is impressive when there are so many drivers to revert to behaviour-driven strategies. SA HiAP’s predominant focus on projects was recognised by the Health Department as limiting the sustainability and systemic effectiveness of HiAP. This has been addressed by it now being supported by a legislative mandate (for example, through PHPA) and a reduced focus on individual projects.

The SA HiAP approach had a small core budget. The approach was constrained in the extent to which it could address the underlying economic and power distribution impact of health, but can be judged to have had a minor impact on longer term population health. Many drivers of health are outside the remit of a state government. HiAP could do more to address health inequities and a promising option may be to partner with citizen groups and others to create a strong constituency for action on more structural determinants of health and a political will for intersectoral policy action for health and equity.

Abbreviations

HiAP:

Health in All Policies

HLA:

Health lens analysis

PHPA:

Public health partner authority

PLM:

Program logic model

SA:

South Australia

SASP:

South Australian strategic plan

SDH:

Social determinants of health

WHO:

World Health Organization

References

  1. Shankardass K, Solar O, Murphy K, Freiler A, Bobbili S, Bayoumi A et al. Health in all policies: a snapshot for Ontario, Results of a Realist-Informed Scoping Review of the Literature. Ontario: 2011. http://www.stmichaelshospital.com/crich/wp-content/uploads/Health-in-All-Policies-A-Snapshot-for-Ontario_FINAL.pdf. Accessed 31 May 2017.

  2. McQueen D, Wismar M, Lin V, Jones C, Davies M. Intersectoral governance for health in all policies: structures, actions and experiences. Copenhagen; 2012. http://www.euro.who.int/en/publications/abstracts/intersectoral-governance-for-health-in-all-policies.-structures,-actions-and-experiences-2012. Accessed 10 Oct 2018

  3. World Health Organization. Moving towards health in all policies: a compilation of experience from Africa, South-East Asia and the Western Pacific. Geneva: 2013. http://www.who.int/social_determinants/publications/9789241506595/en/. Accessed 10 Oct 2018.

  4. Costongs C, Springett J. Towards a framework for the evaluation of health-related policies in cities. Evaluation. 1997. https://0-doi-org.brum.beds.ac.uk/10.1177/135638909700300306.

  5. de Leeuw E, Green G, Dyakova M, Spanswick L, Palmer N. European healthy cities evaluation: conceptual framework and methodology. Health Promot Int. 2015. https://0-doi-org.brum.beds.ac.uk/10.1093/heapro/dav036.

  6. Funnell S, Rogers P. Purposeful Program theory: effective use of the theories of change and logic models. San Francisco: John Wiley & Sons; 2011.

    Google Scholar 

  7. World Health Organization. The Helsinki Statement on Health in All Policies. The 8th Global Conference on Health Promotion, 10–14 June: 2013. http://www.who.int/healthpromotion/conferences/8gchp/8gchp_helsinki_statement.pdf. Accessed 10 Oct 2018.

  8. Baum F, Lawless A, Delany T, MacDougall C, Williams C, Broderick D, et al. Evaluation of health in all policies: concept, theory and application. Health Promot Int. 2014. https://0-doi-org.brum.beds.ac.uk/10.1093/heapro/dau032.

  9. Mayne J. Contribution analysis: coming of age? Evaluation. 2012. https://0-doi-org.brum.beds.ac.uk/10.1177/1356389012451663.

  10. Mayne J. Contribution analysis: Addressing cause and effect. In: Forss K, Marra M, Schwartz R, editors. Evaluating the complex: Attribution, contribution, and beyond. Volume 18; 2011. p. 53–96.

  11. Layder D. New strategies in social research: an introduction and guide. Cambridge, UK: Polity Press; 1993.

    Google Scholar 

  12. Lawless A, Baum F, Delany-Crowe T, MacDougall C, Williams C, McDermott D et al. Developing a framework for a theory-based approach to evaluating policy processes and outcomes: Health in All Policies in South Australia. International Journal of Health Policy and Management. 2017; doi:https://0-doi-org.brum.beds.ac.uk/10.15171/ijhpm.2017.121.

  13. Blamey A, Mackenzie M. Theories of change and realistic evaluation peas in a pod or apples and oranges? Evaluation. 2007. https://0-doi-org.brum.beds.ac.uk/10.1177/1356389007082129.

  14. Pawson R, Tilley N. Realistic Evaluation. London: Sage; 1997.

    Google Scholar 

  15. Baum F, Lawless A, Williams C. Health in All Policies from international ideas to local implementation: policies, systems and organizations. In: Clavier C, de Leeuw E, editors. Health Promotion and the Policy Process: Practical and Critical Theories. Oxford University Press; 2013. p. 188–217.

  16. Lawless A, Williams C, Hurley C, Wildgoose D, Sawford A, Kickbusch I. Health in All Polices: Evaluating the South Australian approach to intersectoral action for health. Can J Public Health. 2012; doi:https://0-doi-org.brum.beds.ac.uk/10.17269/cjph.103.2969.

  17. Pollitt C. Context: what kind of missing link? In: Pollitt C, editor. Context in public policy and management - the missing link? Cheltenham: Edward Elgar; 2013. p. 415–22.

    Google Scholar 

  18. Australian Bureau of Statistics. Catalogue number 3302.0.55.001 - Life Tables, States, Territories and Australia, 2014–2016 - Life expectancy at birth. Canberra 2017. http://www.abs.gov.au/ausstats/abs@.nsf/Previousproducts/3302.0.55.001Main%20Features22014-2016?opendocument&tabname=Summary&prodno=3302.0.55.001&issue=2014-2016&num=&view=. Accessed 13 Dec 2017.

  19. Australian Institute of Health and Welfare. Australia's Health 2016. Australia's health series no 15 cat no AUS 199. Canberra: 2016. https://www.aihw.gov.au/reports/australias-health/australias-health-2016/contents/summary. Accessed 10 Oct 2018.

  20. Baum F, Delany-Crowe T, MacDougall C, Lawless A, van Eyk H, Williams C. Ideas, actors and institutions: lessons from south Australian health in all policies on what encourages other sectors’ involvement. BMC Public Health. 2017. https://0-doi-org.brum.beds.ac.uk/10.1186/s12889-017-4821-7.

  21. van Eyk H, Harris E, Baum F, Delany-Crowe T, Lawless A, MacDougall C. Health in all policies in South Australia - did it promote and enact an equity perspective? Int J Environ Res Public Health. 2017. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph14111288.

  22. Delany T, Lawless A, Baum F, Popay J, Jones L, McDermott D, et al. Health in all policies in South Australia: what has supported early implementation? Health Promot Int. 2016. https://0-doi-org.brum.beds.ac.uk/10.1093/heapro/dav084.

  23. Delany T, Harris P, Williams C, Harris E, Baum F, Lawless A, et al. Health impact assessment in new South Wales & Health in all policies in South Australia: differences, similarities and connections. BMC Public Health. 2014. https://0-doi-org.brum.beds.ac.uk/10.1186/1471-2458-14-699.

  24. Dillman DA. Mail and internet surveys: the tailored design method. New Jersey: John Wiley & Sons; 2007.

    Google Scholar 

  25. Harris P, Williams C, Harris L, Delany T. Health impact assessment and health in all policies in Australia: similarities and differences International Union of Health Promotion and Education World Conference on health promotion Pattaya, Thailand; 2013.

    Google Scholar 

  26. Government of South Australia. Great start – Strong foundations – Powerful learners: A Numeracy and Literacy Strategy from birth to 18. Adelaide: 2013. https://www.education.sa.gov.au/sites/g/files/net691/f/numeracy-literacy-strategy-birth-to-18.pdf?v=1462763791 . Accessed 10 Oct 2018.

  27. Feinstein L, Sabates R, Anderson T, Sorhaindo A, Hammond C. What are the effects of education on health? Report on measuring the effects of education and health and civic engagement. In proceedings of the Copenhagen symposium. Copenhagen. 2006. pp. 171–340.

  28. DeWalt DA, Berkman ND, Sheridan S, Lohr KN, Pignone MP. Literacy and health outcomes: a systematic review of the literature. J Gen Intern Med. 2004;19(12):1228–39.

  29. Spurr B. Enhancing the Overseas Student Experience in Adelaide (the Spurr Report) Cited in: Government of South Australia. International Student Health and Wellbeing: A Health Lens Project Supporting South Australian Vocational Education & Training Sector International Students Final Report https://www.sahealth.sa.gov.au/wps/wcm/connect/c6a74e00408cbd439e18be222b2948cf/International+Students+Health+Lens+Project-Final+Report-PHCS-HiAP-20130730.pdf?MOD=AJPERES&CACHEID=ROOTWORKSPACE-c6a74e00408cbd439e18be222b2948cf-lSjWj2u. Accessed 10 Oct 2018.

  30. Spoehr J. Stormy times: Living with uncertainty [online]. Griffith Review. 2017;55:19–31.

  31. Barton H, Tsourou C. Healthy Urban Planning: Taylor & Francis; 2013.

  32. Litman T. Transportation and public health. Annu Rev Public Health. 2013. https://0-doi-org.brum.beds.ac.uk/10.1146/annurev-publhealth-031912-114502.

  33. Bowen KJ, Parry M. The evidence base for linkages between green infrastructure, public health and economic benefit. Paper prepared for the project Assessing the Economic Value of Green Infrastructure: 2015. Accessed 10 Oct 2018.

  34. Gascon M, Triguero-Mas M, Martínez D, Dadvand P, Rojas-Rueda D, Plasència A et al. Residential green spaces and mortality: A systematic review. Environment International. 2016; https://0-doi-org.brum.beds.ac.uk/10.1016/j.envint.2015.10.013.

  35. Australian Bureau of Statistics. Catalogue number 4364.0.55.001 - National Health Survey: First Results, 2014–15. Canberra: 2016. http://www.abs.gov.au/ausstats/abs@.nsf/mf/4364.0.55.001. Accessed 10 Oct 2018.

  36. Newman L, Ludford I, Williams C, Herriot M. Applying health in all policies to obesity in South Australia. Health Promot Int. 2014. https://0-doi-org.brum.beds.ac.uk/10.1093/heapro/dau064.

  37. Leppo K, Ollila E, Peña S, Wismar M, Cook S. Lessons for policy-makers. In: Leppo K, Ollila E, Peña S, Wismar M, Cook S, editors. Health in All Policies: Seizing opportunities, implementing policies. Helsinki: Ministry of Social Affairs and Health Finland, National Institute for Health and Welfare, and the European Observatory for Health Systems and Policies; 2013. p. 325–338.

  38. Guglielmin M, Muntaner C, O’Campo P, Shankardass K. A scoping review of the implementation of health in all policies at the local level. Health Policy. 2018; https://0-doi-org.brum.beds.ac.uk/10.1016/j.healthpol.2017.12.005.

  39. Labonté R. From mid-level policy analysis to macro-level political economy: Comment on “Developing a framework for a program theory-based approach to evaluating policy processes and outcomes: Health in All Policies in South Australia.”. Int J Health Policy Manag. 2018; doi:https://0-doi-org.brum.beds.ac.uk/10.15171/IJHPM.2018.12.

  40. Commission on the Social Determinants of Health. Closing the gap in a generation: Health equity through action on the social determinants of health. Geneva: 2008. http://www.who.int/social_determinants/thecommission/finalreport/en/index.html. Accessed 25 Jan 2017.

  41. Rose G. Sick individuals and sick populations. Int J Epidemiol. 2001. https://0-doi-org.brum.beds.ac.uk/10.1093/ije/30.3.427.

  42. Delany T, Baum F, Lawless A, MacDougall C, Harris E, McDermott D. Creating a burden of evidence to consider the impact of Health in All Policies: A program logic approach. In: International Political Science Association World Congress: 2014; Montreal; 2014.

  43. Baum F, Graycar A, Delany-Crowe T, de Leeuw E, Bacchi C, Popay J, et al. Understanding Australian policies on public health using social and political science theories: reflections from an Academy of the Social Sciences in Australia workshop. Health Promot Int. 2018. https://0-doi-org.brum.beds.ac.uk/10.1093/heapro/day014.

  44. Baum F, Lawless A, MacDougall C, Delany T, McDermott D, Harris E, et al. New norms new policies: did the Adelaide thinkers in residence scheme encourage new thinking about promoting wellbeing and health in all policies? Soc Sci Med. 2015. https://0-doi-org.brum.beds.ac.uk/10.1016/j.socscimed.2015.10.031.

  45. DeWalt DA, Berkman ND, Sheridan S, Lohr KN, Pignone MP. Literacy and health outcomes. J Gen Intern Med. 2004. https://0-doi-org.brum.beds.ac.uk/10.1111/j.1525-1497.2004.40153.x.

  46. Robertson R. Using information to promote healthy behaviours. London; 2008. https://www.kingsfund.org.uk/sites/default/files/field/field_document/information-promote-healthy-behaviours-kicking-bad-habits-supporting-paper-ruth-robertson.pdf. Accessed 10 Oct 2018

  47. World Health Organization. Interventions on diet and physical activity: What works - Summary report. Geneva: 2009. http://www.who.int/dietphysicalactivity/whatworks/en/. Accessed 10 Oct 2018.

  48. Sacks G, Swinburn B, Lawrence M. Obesity policy action framework and analysis grids for a comprehensive policy approach to reducing obesity. Obes Rev. 2009. https://0-doi-org.brum.beds.ac.uk/10.1111/j.1467-789X.2008.00524.x.

  49. Helps YL, Moller J, Kowanko I, Harrison JE, O’Donnell K, de Crespigny C. Aboriginal people travelling well: issues of safety, transport and health Canberra: 2008. https://infrastructure.gov.au/roads/safety/publications/2008/RSRG_1.aspx. Accessed 10 Oct 2018.

    Google Scholar 

  50. Watson M, Elliott P, Kinsella C, Wilson S. Aboriginal road safety report. Adelaide: 1997. http://www.adac.org.au/resFILE/res31.pdf. Accessed 10 Oct 2018.

  51. Rao M, Prasad S, Adshead F, Tissera H. The built environment and health. The Lancet. 2007; https://0-doi-org.brum.beds.ac.uk/10.1016/S0140-6736(07)61260-4.

  52. Transit Cooperative Research Program. Transit-oriented development and joint development in the United States: a literature review. Washington: 2002. http://onlinepubs.trb.org/onlinepubs/tcrp/tcrp_rrd_52.pdf. Accessed 10 Oct 2018.

Download references

Acknowledgements

We acknowledge the input of all Chief and Associate Investigators who have contributed to the design of this research: Jennie Popay, Elizabeth Harris, Dennis McDermott, Danny Broderick, Ilona Kickbusch, Kevin Buckett, Sandy Pitcher, Andrew Stanley and Deborah Wildgoose. The views expressed in this paper do not necessarily reflect those of the South Australian Government.

Funding

This work was funded by the Australian National Health and Medical Research Council (Grant number 1027561) and the funding body played no role in the design of the study, the collection, analysis of interpretation of the data or in writing the manuscript.

Availability of data and materials

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

FB designed the research, was involved in data collection and analysis, wrote the first and subsequent drafts of this paper; TDC was involved in data collection and analysis and commented on drafts of the paper; CM was involved in the design, data collection and analysis and commented on drafts of the paper; HVE was involved in some data collection and analysis and contributed to writing and revising the drafts; AL was involved in the design, data collection and analysis and commented on drafts of the paper; CW was involved in the design and commented on drafts of the paper; MM was involved in the design and commented on drafts of the paper. All approved the final version for submission.

Corresponding author

Correspondence to Fran Baum.

Ethics declarations

Ethics approval and consent to participate

All data collection activities received prior approval from the Flinders University Social & Behavioural Research Ethics Committee (Project number 5518), and the SA Health Human Research Ethics Committee (Reference number HREC/12/SAH/74). Written consent was obtained from all interviewees prior to interview, including consent to record and transcribe their interview. Interviewees were also offered the opportunity to review and revise the transcripts of their interviews.

Consent for publication

Not applicable.

Competing interests

CM is an Associate Editor of BMC-Public Health. CW is employed by the South Australian Government and manages the Health in All Policies initiative. There are no other competing interests from the authors.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baum, F., Delany-Crowe, T., MacDougall, C. et al. To what extent can the activities of the South Australian Health in All Policies initiative be linked to population health outcomes using a program theory-based evaluation?. BMC Public Health 19, 88 (2019). https://0-doi-org.brum.beds.ac.uk/10.1186/s12889-019-6408-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12889-019-6408-y

Keywords