Synthetic Controls with Multiple Outcomes

Does China’s carbon emission trading scheme reduce CO[formula omitted] emissions? Comprehensive evaluation from synthetic control method using lasso

This paper examines the synthetic control method in contrast to commonly used difference‐in‐differences (DiD) estimation, in the context of a re‐evaluation of a pay‐for‐performance (P4P) initiative, the Advancing Quality scheme. The synthetic control method aims to estimate treatment effects by constructing a weighted combination of control units, which represents what the treated group would have experienced in the absence of receiving the treatment. While DiD estimation assumes that the effects of unobserved confounders are constant over time, the synthetic control method allows for these effects to change over time, by re‐weighting the control group so that it has similar pre‐intervention characteristics to the treated group.We extend the synthetic control approach to a setting of evaluation of a health policy where there are multiple treated units. We re‐analyse a recent study evaluating the effects of a hospital P4P scheme on risk‐adjusted hospital mortality. In contrast to the original DiD analysis, the synthetic control method reports that, for the incentivised conditions, the P4P scheme did not significantly reduce mortality and that there is a statistically significant increase in mortality for non‐incentivised conditions. This result was robust to alternative specifications of the synthetic control method. © 2015 The Authors. Health Economics published by John Wiley & Sons Ltd.

HIV service delivery programs are some of the largest funded public health programs in the world. Timely, efficient evaluation of these programs can be enhanced with methodologies designed to estimate the effects of policy. We propose using the synthetic control method (SCM) as an implementation science tool to evaluate these HIV programs. SCM, introduced in econometrics, shows increasing utility across fields. Key benefits of this methodology over traditional design-based approaches for evaluation stem from directly approximating pre-intervention trends by weighting of candidate non-intervention units. We demonstrate SCM to evaluate the effectiveness of a public health intervention targeting HIV health facilities with high numbers of recent infections on trends in pre-exposure prophylaxis (PrEP) enrollment. This test case demonstrates SCM's feasibility for effectiveness evaluations of site-level HIV interventions. HIV programs collecting longitudinal, routine service delivery data for many facilities, with only some receiving a time-specified intervention, are well-suited for evaluation using SCM.

Quasi-experimental methods increasingly are used to evaluate the impacts of conservation interventions by generating credible estimates of counterfactual baselines. These methods generally require large samples for statistical comparisons, presenting a challenge for evaluating innovative policies implemented within a few pioneering jurisdictions. Single jurisdictions often are studied using comparative methods, which rely on analysts’ selection of best case comparisons. The synthetic control method (SCM) offers one systematic and transparent way to select cases for comparison, from a sizeable pool, by focusing upon similarity in outcomes before the intervention. We explain SCM, then apply it to one local initiative to limit deforestation in the Brazilian Amazon. The municipality of Paragominas launched a multi-pronged local initiative in 2008 to maintain low deforestation while restoring economic production. This was a response to having been placed, due to high deforestation, on a federal “blacklist” that increased enforcement of forest regulations and restricted access to credit and output markets. The local initiative included mapping and monitoring of rural land plus promotion of economic alternatives compatible with low deforestation. The key motivation for the program may have been to reduce the costs of blacklisting. However its stated purpose was to limit deforestation, and thus we apply SCM to estimate what deforestation would have been in a (counterfactual) scenario of no local initiative. We obtain a plausible estimate, in that deforestation patterns before the intervention were similar in Paragominas and the synthetic control, which suggests that after several years, the initiative did lower deforestation (significantly below the synthetic control in 2012). This demonstrates that SCM can yield helpful land-use counterfactuals for single units, with opportunities to integrate local and expert knowledge and to test innovations and permutations on policies that are implemented in just a few locations.

Trade liberalization and child mortality: A Synthetic Control Method

Often considered as an alternative to difference-in-differences methods especially in situations where only aggregated data is available, the synthetic control method offers a data-driven procedure for selecting a comparison unit in comparative case studies, called the “synthetic unit.” It is constructed as a weighted combination of control units, also known as the “donor pool” and aims to best replicate the behavior of the treated unit during the pre-treatment period. A first section introduces the method, while the next presents a result on the bias of the estimator that colors the intuition behind it. This chapter also explores the use cases of the synthetic control method as well as the potential pitfalls. Inference methods for the synthetic control are presented. The seminal article by Abadie et al. (2010) is reproduced. A final section discusses extensions to cases where multiple units are treated.

Uncertainty quantification is a fundamental problem in the analysis and interpretation of synthetic control (SC) methods. We develop conditional prediction intervals in the SC framework, and provide conditions under which these intervals offer finite-sample probability guarantees. Our method allows for covariate adjustment and non-stationary data. The construction begins by noting that the statistical uncertainty of the SC prediction is governed by two distinct sources of randomness: one coming from the construction of the (likely misspecified) SC weights in the pre-treatment period, and the other coming from the unobservable stochastic error in the post-treatment period when the treatment effect is analyzed. Accordingly, our proposed prediction intervals are constructed taking into account both sources of randomness. For implementation, we propose a simulation-based approach along with finite-sample-based probability bound arguments, naturally leading to principled sensitivity analysis methods. We illustrate the numerical performance of our methods using empirical applications and a small simulation study. Python, R and Stata software packages implementing our methodology are available.

When there are multiple outcome series of interest, Synthetic Control analyses typically proceed by estimating separate weights for each outcome. In this paper, we instead propose estimating a common set of weights across outcomes, by balancing either a vector of all outcomes or an index or average of them. Under a low-rank factor model, we show that these approaches lead to lower bias bounds than separate weights, and that averaging leads to further gains when the number of outcomes grows. We illustrate this via a re-analysis of the impact of the Flint water crisis on educational outcomes.

Over the past decade, international development assistance for health has slowed. As donors seek to increase domestic cofinancing and ultimately transition countries from donor aid dependence, COVID-19 has severely constrained public budgets. The evaluation of sustainability and longer-term impacts of donor withdrawal is increasingly important. We assess vaccination coverage and post-neonatal mortality to estimate country performance of these outcomes among countries that no longer received assistance from Gavi, the Vaccine Alliance (Gavi) between 2000 and 2018. Using data from all countries receiving Gavi support between 2000 and 2020, we employed a synthetic control method to generate a pre-transition counterfactual with the same characteristics as the observation of interest to predict a future that empirically never existed. The synthetic unit is constructed from the weighted average of other units with good fit to the unit of interest before transition but did not transition. We found substantial heterogeneity after transitioning from Gavi assistance. China, Guyana, and Turkmenistan overperformed their expected coverage rates; Albania, Bhutan, China, Guyana, and Turkmenistan maintained coverage over 90%; and Bosnia and Herzegovina and Ukraine reported precipitous drop-offs that fell well below their synthetic controls. We also observed a vaccination coverage decline in 2020 for several countries after transitioning and most synthetic controls, which we attribute to COVID-19-related service disruptions. We recommend that Gavi adjust its transition model to systematically assess contextual externalities and risk. In addition, countries that no longer receive Gavi assistance can leverage technical assistance and communities of practice to mutually assist each other and other countries advancing toward transition. This could also foster intracountry accountability after transition. We also recommend that Gavi systematize post-transition assessments and evaluations that leverage the expertise and experience of graduated countries to encourage cross-learning.

ObjectivesTo evaluate the effectiveness of localised Tier 3 restrictions, implemented in England in December 2020, on reducing COVID-19 hospitalisations compared with less stringent Tier 2 measures and the variations by...

Background: This paper investigates the impact of Ecuador’s Ley Orgánica de Educación Superior (LOES), enacted in 2010, which mandated comprehensive reforms in higher education. Objective: The objective of this study is to determine whether the implementation of the LOES has led to a significant increase in academic publications in Ecuador, thereby contributing to sustainable educational practices and aligning with the Sustainable Development Goals (SDGs). In this context, sustainability refers to the ongoing capacity of higher education institutions to generate research that contributes to the long-term development of knowledge, aligns with global sustainability goals, and ensures educational practices that are responsive to societal needs. Methods: Using a synthetic control method, this study evaluates the impact of the LOES on the number of academic publications recorded in Scopus. A synthetic control consisting of data from countries with similar educational and economic profiles serves as a counterfactual to Ecuador’s situation. Results: Before the LOES, the average annual number of publications was stable at 253.4 per year. Post-2010, this number surged, reaching an average of 3645.84 publications per year, representing a 358.41% increase. By 2023, the actual number of publications in Ecuador rose to approximately 6584, significantly surpassing the synthetic control estimate of 1431 publications. Conclusions: The findings confirm that the LOES has been effective in increasing research output and advancing educational sustainability in Ecuador. This analysis not only contributes to the literature on the impact of educational policies but also demonstrates the utility of synthetic control methods in assessing policy effectiveness, underscoring the potential of targeted interventions to promote academic excellence and sustainability in education.

In recent years, many efforts have been made to bring quantitative and qualitative methods into dialogue. This article also moves in that direction. However, in contrast to most works, the present attempt does not concern the large-N/small-N issue but focuses instead on the sole single case study framework. Within this framework, two counterfactual methods, the historical counterfactual method, the qualitative one, and the synthetic control method, the quantitative one, have gained great importance without however meeting. This paper aims to advance mixed-methods research by bridging the gap between these two approaches. More precisely, it has assessed whether these two methods can be used together to understand what would have happened in a single case Z in the absence of an event X. The case study of the impact of Thatcher’s election on the UK pension system is then presented as an example of the joint use of the two methods.

Did Mali Benefit from Joining the CFA Zone? An Analysis Using the Synthetic Control Method.

In order to develop green finance and realize the coordinated development of the environment and economy, China established green finance reform and innovation pilot zones in 2017. Green innovation has problems such as low financing utilization rate and lack of market competitiveness. The green finance pilot policies (GFPP) based on government management provide solutions to these problems. It is of great significance to measure and provide feedback on the implementation effect of GFPP in China for policy-making and green development. This article focuses on the influence of the construction of GFPP by using the five pilot zones as the study area and constructs the green innovation level indicator. Based on the synthetic control method, it chooses provinces that do not carry out the pilot policy as a control group. After that, assign weights to the control region to fit a synthetic control group with resembling characteristics to simulate the five pilot provinces without implementing the policy. Then, compare it with its current policy effect and highlight the policy implementation effect on green innovation. The placebo test and robustness test were conducted to prove the reliability of the conclusions. The results show that since the implementation of GFPP, the level of green innovation in the five pilot cities has shown an overall rising trend. Furthermore, we found that the balance of credit and investment in science and technology has a negative moderating effect on the implementation of GFPP, while the per capita GDP has a significant positive moderating effect.

This paper studies whether research universities can boost regional economic development through an exogenous shock of a forced relocation of a research university in China. We analyze the development in the treated regions compared with a set of control regions that are created using the synthetic control method and find that research universities can have negative effects on local economic development. We then perform a series of robustness checks. Our main results carry through. By employing a more exogenous shock and more reliable identification strategies, our study provides evidence that research universities do not necessarily promote regional economic development.