Helen van Eyk

Evaluating Healthcare Reform: The Challenge of Evaluating Changing Policy Environments

Evaluators in the health sector struggle to develop effective mechanisms with which to evaluate healthcare reform programmes. Evaluation of these programmes is difficult because they are entangled in complex, inherently political processes and often shift away from their original aims and objectives as a result of policy changes within the health system. This article describes the evaluation of a healthcare reform process occurring in South Australia and discusses some of the methodological challenges encountered in evaluating a case study of continuous change with shifting policy objectives. It describes how the evaluators have attempted to address these challenges through a flexible and dynamic action-research approach. The article concludes by discussing the role of the evaluator, as intelligent observer, in reflecting on and documenting change in the highly political and complex field of healthcare reform.

Ideas, actors and institutions: lessons from South Australian Health in All Policies on what encourages other sectors’ involvement

BackgroundThis paper examines the extent to which actors from sectors other than health engaged with the South Australian Health in All Policies (HiAP) initiative, determines why they were prepared to do so and explains the mechanisms by which successful engagement happened. This examination applies theories of policy development and implementation.MethodsThe paper draws on a five year study of the implementation of HiAP comprising document analysis, a log of key events, detailed interviews with 64 policy actors and two surveys of public servants.ResultsThe findings are analysed within an institutional policy analysis framework and examine the extent to which ideas, institutional factors and actor agency influenced the willingness of actors from other sectors to work with Health sector staff under the HiAP initiative. In terms of ideas, there was wide acceptance of the role of social determinants in shaping health and the importance of action to promote health in all government agencies. The institutional environment was initially supportive, but support waned over the course of the study when the economy in South Australia became less buoyant and a health minister less supportive of health promotion took office. The existence of a HiAP Unit was very helpful for gaining support from other sectors. A new Public Health Act offered some promise of institutionalising the HiAP approach and ideas. The analysis concludes that a key factor was the operation of a supportive network of public servants who promoted HiAP, including some who were senior and influential.ConclusionsThe South Australian case study demonstrates that despite institutional constraints and shifting political support within the health sector, HiAP gained traction in other sectors. The key factors that encouraged the commitment of others sectors to HiAP were the existence of a supportive, knowledgeable policy network, political support, institutionalisation of the ideas and approach, and balancing of the economic and social goals of government.

Developing a Framework for a Program Theory-Based Approach to Evaluating Policy Processes and Outcomes: Health in All Policies in South Australia

Background: The importance of evaluating policy processes to achieve health equity is well recognised but such evaluation encounters methodological, theoretical and political challenges. This paper describes how a program theorybased evaluation framework can be developed and tested, using the example of an evaluation of the South Australian Health in All Policies (HiAP) initiative. Methods: A framework of the theorised components and relationships of the HiAP initiative was produced to guide evaluation. The framework was the product of a collaborative, iterative process underpinned by a policy-research partnership and drew on social and political science theory and relevant policy literature. Results: The process engaged key stakeholders to capture both HiAP specific and broader bureaucratic knowledge and was informed by a number of social and political science theories. The framework provides a basis for exploring the interactions between framework components and how they shape policy-making and public policy. It also enables an assessment of HiAP’s success in integrating health and equity considerations in policies, thereby laying a foundation for predicting the impacts of resulting policies. Conclusion: The use of a program theory-based evaluation framework developed through a consultative process and informed by social and political science theory has accommodated the complexity of public policy-making. The framework allows for examination of HiAP processes and impacts, and for the tracking of contribution towards distal outcomes through the explicit articulation of the underpinning program theory.

Health in All Policies in South Australia—Did It Promote and Enact an Equity Perspective?

Mobilising cross-sectoral action is helpful in addressing the range of social determinants that contribute to health inequities. The South Australian Health in All Policies (SA HiAP) approach was implemented from 2007 to stimulate cross-sector policy activity to address the social determinants of health to improve population wellbeing and reduce health inequities. This paper presents selected findings from a five year multi-methods research study of the SA HiAP approach and draws on data collected during interviews, observation, case studies, and document analysis. The analysis shows that SA HiAP had dual goals of facilitating joined-up government for co-benefits (process focus); and addressing social determinants of health and inequities through cross-sectoral policy activity (outcomes focus). Government agencies readily understood HiAP as providing tools for improving the process of intersectoral policy development, while the more distal outcome-focused intent of improving equity was not well understood and gained less traction. While some early rhetorical support existed for progressing an equity agenda through SA HiAP, subsequent economic pressures resulted in the government narrowing its priorities to economic goals. The paper concludes that SA HiAP’s initial intentions to address equity were only partially enacted and little was done to reduce inequities. Emerging opportunities in SA, and internationally, including the UN Sustainable Development Goals, may revive interest in addressing equity.