Staggered difference-in-differences in Python: Callaway–Sant'Anna csdid

Policies rarely start everywhere at once. Minimum-wage increases, school reforms and new regulations are adopted by different states or firms in different years. The classic difference-in-differences regression with group and time fixed effects was long used for such staggered designs. When treatment effects differ across adopters or change with time since adoption, that regression can give a misleading answer. The Callaway–Sant'Anna estimator avoids the problem by building every comparison from units that are not yet affected.

What staggered difference in differences estimates

Group units by the period in which they are first treated; this is their cohort g. Callaway and Sant'Anna (2021) define a separate effect for every cohort and period, the group-time average treatment effect on the treated:

ATT(g, t) = E[Y_t − Y_{g−1} | G = g] − E[Y_t − Y_{g−1} | control units],    t ≥ g

Each ATT(g, t) is a two-group, two-period DiD. It compares cohort g's change from its last untreated period g − 1 to period t with the same change among clean controls: units never treated, or units not yet treated by period t. Treatment is assumed to be absorbing, trends are assumed parallel for each cohort and its controls, and outcomes do not react before adoption. With covariates, parallel trends need only hold conditional on them, and the control change can be estimated by outcome regression, inverse-probability weighting or the doubly robust combination of Sant'Anna and Zhao (2020), which is consistent if either model is right.

The many ATT(g, t) are then averaged into a few summaries. A simple average weights the post-treatment cells by cohort size. A dynamic (event-time) average gives the effect e periods after adoption. Group averages give each cohort's mean effect and calendar averages each period's. Cells before adoption act as placebo tests: under parallel trends they should be zero.

Why not one two-way fixed-effects (TWFE) regression? Goodman-Bacon (2021) shows that the TWFE coefficient is a weighted average of every two-group, two-period comparison in the data. Some of them compare later adopters with earlier adopters that are already treated. If the earlier adopters' effects are still growing, that growth is subtracted from the later adopters' effect. De Chaisemartin and D'Haultfœuille (2020) show that the TWFE coefficient weights the group-time effects with weights that can be negative, so it can even be negative when every effect is positive. Neither problem arises when the effect is the same for all cohorts and periods, or when everyone adopts at the same date.

In OpenEconometrics

oe.csdid estimates all ATT(g, t) and their aggregations. Pass the unit identifier as group, the period as time and each unit's first treated period as treatment_time, missing (NaN) for never-treated units. The call shape is oe.csdid(data=df, y="y", group="county", time="year", treatment_time="first_treat", x=None, method="dr", control="never", base="varying"). It corresponds to the community Stata command csdid, for example csdid y, ivar(county) time(year) gvar(first_treat) (where never-treated units are coded 0 in gvar), and to Stata's hdidregress and xthdidregress.

• method="dr" (doubly robust, the default), "ipw" or "reg" choose how covariates in x adjust the control change; covariates enter at their base-period values. Without covariates the three coincide.
• control="never" or "notyet" chooses the controls. base="varying" compares pre-treatment periods with the period just before; "universal" uses g − 1 throughout.
• result.extra["simple"], ["dynamic"], ["group"] and ["calendar"] hold the aggregations; result.tests["pretrends"] is a Wald test that every pre-treatment ATT(g, t) is zero.
• Standard errors come from influence functions clustered by unit, with z inference. The aggregations include the uncertainty of the estimated cohort shares. bootstrap_reps= with seed= and uniform=True adds multiplier-bootstrap simultaneous bands.
• sample="unbalanced" and "repeated_cross_section" extend the default balanced-panel design; anticipation= shifts the base period for known anticipation.

For diagnosis, oe.bacon decomposes a TWFE estimate into its two-by-two comparisons, and oe.heterodid offers the imputation (model="bjs") and Sun–Abraham (model="sunab") estimators; see the heterogeneous DiD guide. The treatment-effects family guide gives the csdid formulas.

Supported options

Options of oe.csdid

Options of oe.heterodid

oe.bacon

Stata command: none listed.

Generated from oe.capabilities() of openecon 0.3.20a1. Stata names identify the corresponding workflow, not numerical parity.

Example

The script simulates 300 counties observed from 2010 to 2017: 100 adopt a policy in 2012, 100 in 2014 and 100 never do. The effect grows with exposure. It is 0.5 in the first treated year and rises by 0.5 each year after, the same for both cohorts. Early adopters have a higher outcome level, which differencing removes, and all counties share a trend of 0.2 a year. Trends are parallel by construction. Averaged over the ten post-treatment cohort-year cells, which carry equal weight because the cohorts are the same size, the true simple ATT is 1.55. The script also fits the TWFE regression with oe.didregress for comparison.

import numpy as np
import pandas as pd
import openecon as oe


def simulate(n_per_cohort=100, seed=5):
    """Counties observed 2010-2017; cohorts adopt in 2012, in 2014 or never.
    The effect grows with exposure: 0.5 in the first treated year, +0.5 each year after."""
    rng = np.random.default_rng(seed)
    cohorts = [2012] * n_per_cohort + [2014] * n_per_cohort + [np.nan] * n_per_cohort
    rows = []
    for i, g in enumerate(cohorts):
        unit_effect = rng.normal() + (0.5 if g == 2012 else 0.0)   # early adopters differ in levels
        for t in range(2010, 2018):
            effect = 0.5 * (t - g + 1) if t >= g else 0.0          # never treated: g is NaN, t >= g is False
            y = unit_effect + 0.2 * (t - 2010) + effect + rng.normal()
            rows.append((i, t, g, int(t >= g), y))
    return pd.DataFrame(rows, columns=["county", "year", "first_treat", "treated", "y"])


if __name__ == "__main__":
    df = simulate()
    cs = oe.csdid(data=df, y="y", group="county", time="year", treatment_time="first_treat")
    print(cs.summary())
    print("Event time  Estimate  Std. error  (true effect)")
    for row in cs.extra["dynamic"]:
        e = row["label"]
        print(f"{e:>10}  {row['estimate']:8.3f}  {row['std_error']:10.3f}  ({0.5 * (e + 1) if e >= 0 else 0.0:.1f})")
    twfe = oe.didregress(data=df, y="y", treatment="treated", group="county", time="year").coefficients[0]
    print(f"TWFE didregress ATET: {twfe.estimate:.3f}, 95% CI [{twfe.ci_low:.3f}, {twfe.ci_high:.3f}]")

With openecon 0.3.20a1 from PyPI the script prints:

Callaway-Sant'Anna group-time ATT — y
Observations: 2400  |  Covariance: robust  |  Confidence: 95%

Term             Estimate  Std. error          z     P>|stat|    CI lower  CI upper
--------------  ---------  ----------  ---------  -----------  ----------  --------
[Cohort 2012]
ATT(2012,2011)   0.306393     0.20459     1.4976     0.134237  -0.0945947  0.707382
ATT(2012,2012)   0.384907    0.202665    1.89922     0.057535  -0.0123098  0.782124
ATT(2012,2013)   0.994143    0.221661    4.48497   7.2925e-06    0.559695   1.42859
ATT(2012,2014)    1.37992    0.184045    7.49771  6.49448e-14     1.01919   1.74064
ATT(2012,2015)    1.80318    0.207295    8.69863  3.35918e-18     1.39689   2.20947
ATT(2012,2016)    2.18744    0.208667    10.4829  1.03505e-25     1.77846   2.59642
ATT(2012,2017)    3.01146    0.200425    15.0254  5.00817e-51     2.61863   3.40428
[Cohort 2014]
ATT(2014,2011)    0.58054    0.195042    2.97649   0.00291566    0.198266  0.962815
ATT(2014,2012)  -0.115833    0.205283  -0.564261     0.572577    -0.51818  0.286514
ATT(2014,2013)  -0.150071    0.200285  -0.749287     0.453684   -0.542623  0.242481
ATT(2014,2014)   0.467892    0.208912    2.23966    0.0251129   0.0584322  0.877351
ATT(2014,2015)   0.864594    0.215235    4.01697  5.89502e-05    0.442741   1.28645
ATT(2014,2016)    1.30615    0.202807    6.44036  1.19194e-10    0.908656   1.70365
ATT(2014,2017)    2.15605    0.207164    10.4074  2.29337e-25     1.75001   2.56208

n_groups: 300  |  n_periods: 8  |  n_cohorts: 2  |  simple_att: 1.45557
simple_att_se: 0.11476
Pre-treatment ATT(g,t) = 0 (parallel trends before treatment): chi2(4) = 10.6526, p = 0.03076
Event time  Estimate  Std. error  (true effect)
        -3     0.581       0.195  (0.0)
        -2    -0.116       0.205  (0.0)
        -1     0.078       0.147  (0.0)
         0     0.426       0.143  (0.5)
         1     0.929       0.132  (1.0)
         2     1.343       0.138  (1.5)
         3     1.980       0.147  (2.0)
         4     2.187       0.209  (2.5)
         5     3.011       0.200  (3.0)
TWFE didregress ATET: 1.021, 95% CI [0.850, 1.193]

Reading the output

• The group-time effects track the truth. For the 2012 cohort the post-treatment cells rise from 0.385 in 2012 to 3.011 in 2017, against true values of 0.5 to 3.0. Each cell has a standard error of about 0.2, so individual cells are noisy.
• The simple ATT is close to 1.55. It is 1.456 with a standard error of 0.115, less than one standard error from the true value.
• The event-time profile recovers the dynamics. The dynamic aggregation gives 0.426, 0.929, 1.343 and 1.980 for the first four years after adoption, against 0.5, 1.0, 1.5 and 2.0. Event times 4 and 5 rest on the 2012 cohort alone.
• TWFE understates the effect. The TWFE ATET is 1.021 with an interval from 0.850 to 1.193, which excludes 1.55. Part of the comparison uses the 2012 cohort, already treated and with growing effects, as a control for the 2014 cohort. oe.didregress adds a warning to that effect.
• The pre-trend test rejects, falsely. The four pre-treatment cells give χ²(4) = 10.65 with p = 0.031, driven by ATT(2014,2011) = 0.581. Trends are parallel by construction, so this is a false alarm of the kind a 5% test raises in one sample in twenty. A rejection in real data calls for a closer look, not automatic rejection of the design; a non-rejection is not proof of parallel trends either.

Limitations

• oe.csdid accepts no weights.
• The common oe.predict and oe.margins adapters do not cover oe.csdid; use its own result tables and family helpers.
• oe.heterodid accepts no weights.
• The common oe.predict and oe.margins adapters do not cover oe.heterodid; use its own result tables and family helpers.
• oe.heterodid fits in-memory data only; it has no out-of-core oe.Dataset route.
• Units first treated in or before the first observed period are removed from the estimation sample, and units first treated after the last period count as never treated; both produce a warning.
• The default sample="balanced" requires a balanced panel and raises unbalanced_panel otherwise. sample="unbalanced" uses the units observed in both periods of each comparison, without any correction for attrition, and sample="repeated_cross_section" accepts neither group nor covariates.
• Propensity scores are not trimmed: a control-group propensity within 1e-5 of one raises overlap_violation for method="ipw" and "dr".
• Covariates are taken at the base period of each comparison; the coefficient table's standard errors are analytic even when multiplier bands are requested.
• No numerical parity with Stata is claimed: the inventory's stata_parity_validated flag is False.

References

• Callaway, B. and Sant'Anna, P. H. C. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200–230. doi:10.1016/j.jeconom.2020.12.001
• de Chaisemartin, C. and D'Haultfœuille, X. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110(9), 2964–2996. doi:10.1257/aer.20181169
• Goodman-Bacon, A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2), 254–277. doi:10.1016/j.jeconom.2021.03.014
• Sant'Anna, P. H. C. and Zhao, J. (2020). Doubly robust difference-in-differences estimators. Journal of Econometrics, 219(1), 101–122. doi:10.1016/j.jeconom.2020.06.003