Synthetic control in Python: donor weights, placebos and synthetic DiD

Some policies affect a single unit: one state raises its tobacco tax, one country joins a currency union, one city opens a new metro line. There is no group of treated units to average over, and no single untreated unit is a convincing comparison. The synthetic control method builds the comparison instead. It finds a weighted average of untreated units whose outcomes tracked the treated unit closely before the policy, and uses that average to show what would have happened without it.

What synthetic control estimates

Let unit 1 be treated from period T0 on and units j = 2, …, J + 1 form the donor pool of untreated units. Synthetic control chooses donor weights w_j that reproduce the treated unit's pre-treatment outcomes:

min over w   Σ_{t<T0} ( y_1t − Σ_j w_j y_jt )²     subject to  w_j ≥ 0,  Σ_j w_j = 1

The effect in each later period is the gap between the treated unit and its synthetic version, τ_t = y_1t − Σ_j w_j y_jt, and the average effect on the treated (ATT) averages those gaps. Because the weights are non-negative and sum to one, the synthetic unit is an interpolation of real donors, never an extrapolation, and the weights show exactly which units make up the comparison (Abadie and Gardeazabal 2003; Abadie, Diamond and Hainmueller 2010).

The method rests on a long pre-treatment period that the synthetic unit fits closely. Abadie, Diamond and Hainmueller (2010) motivate it with a factor model: if the synthetic unit matches the treated unit's outcomes over many pre-treatment periods, it also matches the unobserved factor loadings, and the bias of the estimate is small. With few pre-treatment periods or noisy outcomes, close pre-treatment fit can come from fitting noise. Abadie (2021) discusses this and the other conditions the method needs: a donor pool of comparable units that are not affected by the policy, no anticipation, and no large shocks specific to the treated unit after T0.

Inference uses placebos. Each donor is treated in turn as if it had adopted the policy, and the treated unit's estimate is compared with the distribution of placebo estimates. If the real effect is large relative to the placebos, it is unlikely to be chance.

Synthetic difference in differences (SDID, Arkhangelsky, Athey, Hirshberg, Imbens and Wager 2021) combines the two ideas. It reweights donors, like synthetic control, but allows a constant level difference between the treated unit and its synthetic version, like difference in differences. It also weights the pre-treatment periods, emphasizing those that best predict the donors' post-treatment outcomes. The weights are regularized, so they spread over more donors.

In OpenEconometrics

oe.synthcontrol fits both models: oe.synthcontrol(data=df, y="y", treatment="policy", panel="region", time="year", model="sc"). treatment is a 0/1 indicator that switches on for the treated unit (or units) at one common date and stays on. The capability inventory lists no corresponding Stata command.

• model="sc" (the default) fits the simplex-constrained weights above, matching the treated units' mean pre-treatment outcome path, with no intercept. ridge= adds an explicit ridge penalty; the default is 0.
• model="sdid" profiles out intercepts in separate unit-weight and time-weight problems. The penalties zeta_omega and zeta_lambda default to data-driven values; pass numbers to fix them.
• Uncertainty comes from placebos drawn from the original donors only. With one treated unit every donor is treated in turn; with several, placebo_reps random same-size assignments are drawn using seed. The reported standard error is the dispersion of the placebo effects, and the summary adds a plus-one placebo tail probability for the absolute effect.
• result.extra holds the donor_weights with their donor_labels, the SDID pre_period_weights, the period-by-period gaps in effect_curve, the pre-treatment fit pre_rmspe and every placebo_effects value.

The heterogeneous DiD and synthetic control guide gives the objective functions, default penalties and placebo algorithm.

Supported options

Options of oe.synthcontrol

Generated from oe.capabilities() of openecon 0.3.20a1. Stata names identify the corresponding workflow, not numerical parity.

Example

The script simulates 31 regions from 2000 to 2019 with a two-factor model: a common trend factor and a cycle factor, each region with its own level and factor loadings, plus independent noise with standard deviation 0.5. The treated region's level and loadings are the average of three donors, region01 to region03, so a perfect synthetic control exists in principle. From 2012 on the policy lowers the treated region's outcome by 3.0 in every year; the true ATT is −3.0. The script fits synthetic control and then SDID.

import numpy as np
import pandas as pd
import openecon as oe


def simulate(n_donors=30, seed=4):
    """One treated region and 30 donors, 2000-2019, from a two-factor model.
    The treated region's level and loadings are the average of donors 1, 2 and 3.
    The policy starts in 2012 and lowers the outcome by 3.0 in every later year."""
    rng = np.random.default_rng(seed)
    years = np.arange(2000, 2020)
    f1 = 0.3 * (years - 2000) + rng.normal(scale=0.3, size=years.size)   # common trend factor
    f2 = np.sin((years - 2000) / 2.0)                                      # cycle factor
    level = rng.normal(20, 3, size=n_donors)
    load = rng.uniform(0, 2, size=(n_donors, 2))
    level_t, load_t = level[:3].mean(), load[:3].mean(axis=0)
    rows = []
    for i in range(n_donors + 1):
        a, (m1, m2) = (level_t, load_t) if i == 0 else (level[i - 1], load[i - 1])
        for k, t in enumerate(years):
            d = int(i == 0 and t >= 2012)
            y = a + m1 * f1[k] + m2 * f2[k] - 3.0 * d + rng.normal(scale=0.5)
            rows.append((f"region{i:02d}", t, d, y))
    return pd.DataFrame(rows, columns=["region", "year", "policy", "y"])


if __name__ == "__main__":
    df = simulate()
    sc = oe.synthcontrol(data=df, y="y", treatment="policy", panel="region", time="year", model="sc")
    print(sc.summary())
    weights = sorted(zip(sc.extra["donor_weights"], sc.extra["donor_labels"]), reverse=True)
    print("Donor weights:", ", ".join(f"{label} {w:.3f}" for w, label in weights if w > 0))
    print(f"Pre-treatment RMSPE: {sc.extra['pre_rmspe']:.3f}")
    gaps = sc.extra["effect_curve"][sc.extra["adoption_period_index"]:]
    print("Yearly gaps 2012-2019:", " ".join(f"{g:.2f}" for g in gaps))
    sdid = oe.synthcontrol(data=df, y="y", treatment="policy", panel="region", time="year", model="sdid")
    print()
    print(sdid.summary())

With openecon 0.3.20a1 from PyPI the script prints:

Synthetic control — y
Observations: 620  |  Covariance: nonrobust  |  Confidence: 95%

Term  Estimate  Std. error         z   P>|stat|  CI lower    CI upper
----  --------  ----------  --------  ---------  --------  ----------
ATT   -2.24982     1.09836  -2.04834  0.0405263  -4.40257  -0.0970737
placebo: same-size original-control assignment distribution = -2.24982, p = 0.129
Donor weights: region02 0.341, region21 0.214, region27 0.178, region19 0.109, region18 0.084, region09 0.061, region05 0.012
Pre-treatment RMSPE: 0.276
Yearly gaps 2012-2019: -3.44 -1.54 -2.68 -2.46 -1.58 -2.03 -2.19 -2.09

Synthetic DiD — y
Observations: 620  |  Covariance: nonrobust  |  Confidence: 95%

Term  Estimate  Std. error         z    P>|stat|  CI lower   CI upper
----  --------  ----------  --------  ----------  --------  ---------
ATT   -2.85423     0.95047  -3.00297  0.00267359  -4.71712  -0.991345
placebo: same-size original-control assignment distribution = -2.85423, p = 0.03226

Reading the output

• Synthetic control finds the direction but understates the size. The ATT is −2.250 against a true −3.0. The yearly gaps range from −3.44 in 2012 to −1.54 in 2013 and average −2.25.
• The weights are sparse and not the "true" donors. Seven donors get positive weight, led by region02 with 0.341. Of the three donors that generated the treated region only region02 is used, because other combinations reproduce similar factor loadings. Weights describe a good comparison, not a causal structure.
• Good pre-treatment fit is not proof. The pre-treatment RMSPE is 0.276, smaller than the treated region's own noise standard deviation of 0.5. With 12 pre-treatment years and 30 donors, the weights partly fit that noise, so a close fit does not guarantee that the factor loadings match.
• The placebo test is the honest test here. Three of the 30 placebo effects are at least as large in absolute value as 2.250, so the plus-one tail probability is (3 + 1)/31 = 0.129. The normal-approximation p-value of 0.041 is smaller; it treats the placebo spread, 1.098, as a standard error, which is only an approximation.
• SDID is closer to the truth. It estimates −2.854 with a placebo dispersion of 0.950. No placebo is as large, so the tail probability is 1/31 = 0.032, the smallest possible with 30 donors.

Limitations

• oe.synthcontrol accepts no weights.
• The common oe.predict and oe.margins adapters do not cover oe.synthcontrol; use its own result tables and family helpers.
• oe.synthcontrol fits in-memory data only; it has no out-of-core oe.Dataset route.
• The panel must be balanced on consecutive periods, with one common adoption date, a binary absorbing treatment, at least two pre-treatment periods, more donors than treated units and no always-treated units.
• There are no covariates or predictor variables and no optimization of predictor-importance weights: model="sc" matches the pre-treatment outcome path only.
• Placebo inference assumes the original donors are exchangeable with the treated units and have comparable noise; the normal interval is an approximation under those conditions. Placebos are not filtered by their pre-treatment fit.
• max_work bounds the computational work of the fit and all placebos together, and a singular weight problem raises an error instead of adding hidden regularization.
• Neither a close pre-treatment fit nor an insignificant pre-trend diagnostic verifies the identifying assumptions.
• No numerical parity with Stata is claimed: the inventory's stata_parity_validated flag is False.

References

• Abadie, A. (2021). Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature, 59(2), 391–425. doi:10.1257/jel.20191450
• Abadie, A., Diamond, A. and Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program. Journal of the American Statistical Association, 105(490), 493–505. doi:10.1198/jasa.2009.ap08746
• Abadie, A. and Gardeazabal, J. (2003). The economic costs of conflict: A case study of the Basque Country. American Economic Review, 93(1), 113–132. doi:10.1257/000282803321455188
• Arkhangelsky, D., Athey, S., Hirshberg, D. A., Imbens, G. W. and Wager, S. (2021). Synthetic difference-in-differences. American Economic Review, 111(12), 4088–4118. doi:10.1257/aer.20190159