Header image

S.2.34 Evaluation of policy interventions in complex systems: The example of sugary drinks taxes

Tracks
Room: Waitakere #1 Level 3
Friday, June 19, 2020
5:00 PM - 6:15 PM
Waitakere #1 Level 3

Details

Purpose: This symposium will bring together scientists interested in the challenges of evaluating interventions in complex systems. We will illustrate these challenges by showcasing evaluations of sugary drinks taxes that take a wider perspective, evaluating impacts beyond anticipated, proximal outcomes such as price and purchasing, to reveal the wider societal and economic consequences and impacts on health. We will discuss the challenges of such systemic evaluations, with an emphasis on methods to integrate multiple data sources, analyses and interpretations to draw conclusions on policy impacts and identify generalizable conclusions. Rationale: Sugary drinks taxes take multiple forms and can have a range of impacts on the population, resulting from: changes in formulation, price, volume and consumption; signalling effects via public discourse; impacts on marketing strategies; economic impacts, such as on employment in food and other sectors; and impacts on a range of short, medium and long-term health outcomes. Making sense of such diverse outcomes is essential both to determining the overall impact of such a policy and understanding how it works. Yet, the methods available for undertaking such systemic evaluations remain unclear, in particular how we should integrate data and findings from diverse analyses. Objectives: 1. Bring together scientists interested in evaluations of interventions in complex systems, using sugary drinks taxes as the exemplar 2. Provide three examples of sugary drinks tax evaluations that have attempted to capture systemic impacts to illustrate different aspects of the methodological challenges of such research 3. Identify in particular the challenges of integrating data, findings and conclusions from multiple analyses 4. Discuss the value of taking a systems perspective in policy evaluations and identify the implications for future research and policy. Summary: We will present evaluations of SSB taxes in three countries: the UK, Mexico and South Africa. Each tax has a different design and has been implemented in a different context. This complexity has demanded a range of evaluative methods, yet in each case a mixed-method approach has aimed to account for the intervention as an event in a complex adaptive system, exploring expected and unanticipated impacts. The discussion will focus on commonalities of approach to evaluation, exploring the vital ingredients of a system level evaluation and highlighting methods to integrate findings to draw overall conclusions about impact. Format: Introduction (10 minutes) Paper 1 (15 minutes) Paper 2 (15 minutes) Paper 3 (15 minutes) Discussion (20 minutes)


Speaker

Professor Martin White
Professor Of Population Health Research
Mrc Epidemiology Unit, University Of Cambridge, Uk

Evaluation of the UK Soft Drinks Industry Levy

Abstract

Purpose

To evaluate the systemic impacts of the UK Soft Drinks Industry Levy (SDIL)

Methods

We are undertaking a multi-method, longitudinal study of the impacts of the SDIL over a 6 year period on population health outcomes and across key sectors. Analyses are underpinned by a conceptual map of the systems on which the SDIL has its impacts. Quantitative evaluation uses controlled, interrupted time-series analyses of changes in formulation, price and volume of sugary drinks, purchasing and consumption of drinks, and proximal and distal health outcomes in the short term. Health outcomes will be modelled over 5-15 year time horizons. An economic evaluation is exploring the impacts of the SDIL on food and other industries and on the UK economy. Qualitative studies are evaluating the perceived effects of the levy among key stakeholder groups, including the public, the food industry and policymakers using analysis of parliamentary documents, trade and public media, telephone interviews and focus groups. Interpretation of the findings of all analyses will make use of methods and frameworks for integration

Results

The latest findings from key analyses will be presented, including those on reformulation, changes in volume and price, purchasing and consumption, and economic impacts from one and two years post implementation of the SDIL, as well as key qualitative data. We will focus in particular on the results of integrative analyses and interpretation of findings from the multiple strands of analysis.

Conclusions

Analyses of multiple data sources and outcomes tell different stories about the impacts of the SDIL on people, the food system and society. Making sense of these wide ranging findings as a whole requires a systemic perspective, an underpinning theory of change, a willingness to consider each finding in a wider context, and regularly revisiting a conceptual map so as to draw reasonable causal inference about the impacts of the levy, including the reactions and counteractions of key stakeholders over time.

Prof. Karen Hofman

An assessment of the impacts of the introduction of South Africa’s Health Promotion Levy

Abstract

Purpose

To assess the impacts of South Africa’s recently implemented tax on sugar-sweetened beverages, the Health Promotion Levy, on dietary and other domains.

Methods

We apply a multidisciplinary approach to evaluating the impact of the introduction of the Health Promotion Levy on relevant consumer responses, such as dietary and health outcomes, and on producer responses including reformulation, pricing, and marketing. Qualitative studies interrogate in greater depth the knowledge, attitudes and responses of key demographic groups undergoing lifestyle change to the adoption of this policy. As well as evaluating the impact of the policy on its desired outcomes, we study the nature of the political and legislative processes through documentary analysis of submissions made during parliamentary hearings and calls for comment with an eye to understanding the positions and strategy of those in favour and against the adoption of the levy.

Results

A summary of the integrated research program both qualitative and quantitative will be presented for the first year following legislation of the SSB tax, emphasizing the relevance of our findings to understanding the intermediate pathways to the ultimate dietary and health outcomes.

Conclusions

Precedence from other settings together with local economic modelling evidence played a critical role for national policymakers in Treasury and the Health Department. Following a two year process of consultation by Treasury with simultaneous strong advocacy by civil society, public acceptance was high. However, the political economy played a critical role in a setting with high unemployment and concerns about state capture.  The final design of the policy resulted in higher prices and reformulation of carbonated beverages. Future regulation in South Africa for population level interventions to address commercial determinants of health will need to consider a comprehensive, integrated approach to anticipate interference by industry.

Dr. Shu Wen Ng

Uncovering heterogeneous responses to and impacts of Mexico’s sugary drink tax

Abstract

Purpose

To illustrate how considering different stakeholders’ responses can help uncover potential heterogeneities in impacts of Mexico’s sugary drink tax.

Methods

Multiple data and methods have been used in assessing Mexico’s sugary drink tax. Each approach has its strengths and limitations. We will describe how these data and methods can allow us to evaluate whether and to what extent responses can vary. These include understanding differential firm responses in price pass-through (across locations, beverage types, package sizes and based on firm market share), as well as consumer responses via changes in purchases based on their pre-policy purchase levels and income levels in the shorter term and medium term. We will also describe how structural models that take into account these supply and demand responses, validated from the empirical evidence to date, can be used to test new or higher tax policy designs. Finally, we will show how these empirical and simulated outcomes on purchases can be translated to changes in health outcomes, their resultant healthcare cost savings over time and costs-benefits borne by various stakeholders.

Results

We will present past and new findings to date on the Mexico sugary drink tax on variability in price-pass-through and purchase changes based on characteristics of beverage firms and Mexican households and where they are consistent or divergent with each other. Additionally, we will show the importance of considering issues like simultaneously accounting for supply and demand responses, time horizons of interest, and whose point of view to include in assessing costs and benefits of sugary drink taxes, particularly as they relate to policy decisions.

Conclusions

There is a need to integrate multiple disciplines and use multiple data and methods in assessing how and to what extent various stakeholders respond to sugary drink taxes and in helping inform improvements in tax designs.


Chair

Jean Adams


Discussant

Jean Adams

loading