Regression discontinuity explained: estimating effects at a cutoff

Many decisions follow a rule on a number. A candidate with one vote more than the rival wins the seat. A student who scores 60 gets the scholarship and one who scores 59 does not. A firm with 49 employees escapes a regulation that starts at 50. Units just either side of such a threshold are nearly identical, yet only some of them are treated. A regression discontinuity (RD) design uses that feature of the rule to estimate a causal effect.

This article builds the intuition, works through a simulated example with OpenEconometrics and lists what can go wrong. The method reference documents every option of oe.rdrobust.

The idea: a rule creates a comparison

Thistlethwaite and Campbell (1960) introduced the design in educational psychology, studying students near the threshold for an academic award, a certificate of merit. It became common in economics much later; Lee and Lemieux (2010) survey the applications.

Three ingredients define a sharp design:

• a running variable x, such as a vote margin or a test score;
• a cutoff c;
• a treatment that switches on exactly when x ≥ c.

The effect is the size of the jump in the average outcome at the cutoff:

τ = lim_{x↓c} E[y | x] − lim_{x↑c} E[y | x]

The identifying assumption is continuity: without the treatment, the average outcome would have changed smoothly as x crossed c. Nothing else may jump at the cutoff. Lee and Lemieux (2010) explain why this is often credible. If units cannot control the running variable precisely, chance decides on which side of the threshold they land, so treatment near the cutoff is as good as randomly assigned. RD replaces the "no confounders" assumption discussed in correlation versus causation with an assumption about a single point.

A simulated example: close elections

Suppose party A contests 2,000 seats. Its winning margin at election t, in percentage points, is spread evenly from −50 to 50, and A holds the seat when the margin is 0 or more. The outcome is A's vote share at the next election. In the simulation, the vote share rises smoothly along an S-shaped curve in the margin. Winning adds 5 points at the cutoff, and the effect shrinks by 0.06 points for every point of margin. The noise has a standard deviation of 5 points. The data are synthetic and describe no real election. The code blocks below continue one Python session.

import numpy as np
import pandas as pd
import openecon as oe


def simulate(n=2000, seed=2026):
    """Simulated races: party A wins election t when its vote margin is 0 or more."""
    rng = np.random.default_rng(seed)
    margin = rng.uniform(-50, 50, size=n)      # A's margin at election t, percentage points
    won = (margin >= 0).astype(float)          # A won election t
    effect = 5 - 0.06 * margin                 # effect of winning: 5 points at the cutoff
    share = 50 + 15 * np.tanh(margin / 15) + won * effect + rng.normal(scale=5, size=n)
    return pd.DataFrame({"margin": margin, "won": won, "share": share})


df = simulate()
naive = df.share[df.won == 1].mean() - df.share[df.won == 0].mean()
print(f"Winners minus losers, all races: {naive:.2f}")
print(f"True average effect among winners: {(5 - 0.06 * df.margin[df.won == 1]).mean():.2f}")

rd = oe.rdrobust(data=df, y="share", running="margin", cutoff=0)
print(rd.summary())

With openecon 0.3.20a1 from PyPI, the script prints:

Winners minus losers, all races: 27.48
True average effect among winners: 3.48
Regression discontinuity: sharp local polynomial — share
Observations: 2000  |  Covariance: robust  |  Confidence: 95%

Term            Estimate  Std. error        z    P>|stat|  CI lower  CI upper
--------------  --------  ----------  -------  ----------  --------  --------
Conventional     3.26994     1.20911  2.70442  0.00684241  0.900127   5.63975
Bias-corrected   2.72112     1.20911  2.25051   0.0244164  0.351305   5.09093
Robust           2.72112     1.32984  2.04619   0.0407373  0.114673   5.32756

h_left: 9.30632  |  h_right: 9.30632  |  b_left: 19.2438  |  b_right: 19.2438
n_left: 983  |  n_right: 1017  |  n_h_left: 179  |  n_h_right: 172
n_b_left: 384  |  n_b_right: 377  |  p: 1  |  q: 2
cutoff: 0

Next-election vote share against the winning margin in 2,000 simulated races

The naive comparison fails. Winners' average vote share is 27.48 points above losers'. Most of that gap is not the effect of winning: a party that won by 40 points was already far stronger than one that lost by 40. The true effect at the cutoff is 5.

The RD estimate uses only races near the cutoff. oe.rdrobust chose a bandwidth h of 9.31 points, so only the 179 races just below and the 172 just above the cutoff carry weight. The conventional estimate of the jump is 3.27. The robust bias-corrected estimate is 2.72, with a 95% confidence interval from 0.11 to 5.33 that contains the true 5. In this sample the estimate happens to fall below the truth; a later section repeats the experiment 200 times to show that this is noise rather than bias.

Local linear regression by hand

The estimator fits a separate straight line on each side of the cutoff, using only data near it, and measures the gap between the two lines at c. Imbens and Lemieux (2008) recommend this local linear approach. Within a window of width h on each side, it is one ordinary least squares regression with an interaction:

yᵢ = α + τ Dᵢ + β₁ xᵢ + β₂ Dᵢ xᵢ + uᵢ,   for |xᵢ − c| ≤ h

Dᵢ is 1 for winners, so τ is the jump at the cutoff, and β₂ lets the slope differ on the two sides.

h = 10
near = df[df.margin.abs() <= h].assign(won_x_margin=lambda d: d.won * d.margin)
ll = oe.ols(data=near, y="share", x=["won", "margin", "won_x_margin"], covariance="HC1")
jump = {c.term: c for c in ll.coefficients}["won"]
print(f"OLS on {ll.nobs} races with |margin| <= {h}: jump {jump.estimate:.3f} "
      f"(95% CI {jump.ci_low:.3f} to {jump.ci_high:.3f})")
uniform = oe.rdrobust(data=df, y="share", running="margin", cutoff=0, h=h, kernel="uniform")
print(f"rdrobust with a uniform kernel and h = {h}: {uniform.coefficients[0].estimate:.3f}")

OLS on 380 races with |margin| <= 10: jump 3.925 (95% CI 1.849 to 6.001)
rdrobust with a uniform kernel and h = 10: 3.925

OLS on the 380 races within 10 points of the cutoff gives a jump of 3.925, identical to oe.rdrobust with a uniform kernel. The default triangular kernel gives more weight to the races closest to the cutoff, which is why the same bandwidth gives 3.43 in the next section. The OLS interval is shown for intuition only, because it ignores the bias discussed next.

Bandwidth: the bias–variance trade-off

The bandwidth decides how far from the cutoff the data reach. A narrow window keeps the comparison local but uses few observations. A wide window uses many observations, but the straight lines must then span a curved relationship, and the curvature leaks into the estimated jump.

for h in [2.5, 5, 10, 20, 30, 50]:
    fit = oe.rdrobust(data=df, y="share", running="margin", cutoff=0, h=h)
    c = fit.coefficients[0]                    # the conventional row
    used = fit.metrics["n_h_left"] + fit.metrics["n_h_right"]
    print(f"h = {h:>4}: {c.estimate:6.2f}  [{c.ci_low:6.2f}, {c.ci_high:6.2f}]  races used: {used:.0f}")

h =  2.5:   3.83  [ -1.52,   9.18]  races used: 88
h =    5:   2.56  [ -0.88,   6.01]  races used: 177
h =   10:   3.43  [  1.16,   5.71]  races used: 380
h =   20:   5.34  [  3.77,   6.91]  races used: 800
h =   30:   8.39  [  7.16,   9.63]  races used: 1227
h =   50:  13.62  [ 12.67,  14.56]  races used: 2000

Estimated jump at six bandwidths, with conventional 95% confidence intervals

With h = 2.5, only 88 races are used and the interval runs from −1.52 to 9.18. At h = 30 and h = 50 the intervals are narrow but exclude the truth: the estimates of 8.39 and 13.62 measure the curvature of the vote-share curve as much as the effect of winning.

One sample can mislead, so the next block repeats the simulation 200 times with new seeds.

fixed, optimal = [], []
for seed in range(1, 201):
    sample = simulate(seed=seed)
    for h in [2.5, 5, 10, 20, 30, 50]:
        c = oe.rdrobust(data=sample, y="share", running="margin", cutoff=0, h=h).coefficients[0]
        fixed.append({"h": h, "estimate": c.estimate, "covers": c.ci_low <= 5 <= c.ci_high})
    fit = oe.rdrobust(data=sample, y="share", running="margin", cutoff=0)
    conv, robust = fit.coefficients[0], fit.coefficients[2]
    optimal.append({"h": fit.metrics["h_left"], "estimate": conv.estimate,
                    "conventional covers": conv.ci_low <= 5 <= conv.ci_high,
                    "robust covers": robust.ci_low <= 5 <= robust.ci_high})

summary = pd.DataFrame(fixed).groupby("h").agg(
    mean=("estimate", "mean"), sd=("estimate", "std"),
    rmse=("estimate", lambda e: np.sqrt(((e - 5) ** 2).mean())), coverage=("covers", "mean"))
print(summary.round(2))
print(pd.DataFrame(optimal).mean().round(2).to_string())

       mean    sd  rmse  coverage
h
2.5    5.03  2.24  2.23      0.94
5.0    4.95  1.59  1.59      0.95
10.0   5.24  1.10  1.13      0.94
20.0   6.54  0.75  1.71      0.50
30.0   8.66  0.63  3.71      0.00
50.0  13.20  0.53  8.21      0.00
h                      10.44
estimate                5.22
conventional covers     0.92
robust covers           0.95

The pattern is the classic trade-off. As h grows from 2.5 to 50, the standard deviation of the estimate falls from 2.24 to 0.53, but the average estimate drifts from 5.03 to 13.20. The root mean squared error, which combines both, is smallest at h = 10 among the fixed bandwidths. The default data-driven bandwidth (bwselect="mserd", one bandwidth that minimizes an estimate of the mean squared error) averaged 10.44. Conventional intervals at h = 20 covered the true jump in only half of the samples.

The last two lines show why rdrobust reports a robust interval. At the MSE-optimal bandwidth the bias is of the same order as the standard error, so conventional 95% intervals cover the truth less often than they claim. Calonico, Cattaneo and Titiunik (2014) subtract an estimate of the bias and widen the standard error to account for it. Here the conventional interval covered the true jump in 92% of samples and the robust interval in 95%. With 200 replications, each coverage rate is uncertain by roughly 3 to 4 percentage points, but the direction matches the theory.

In practice: report the robust interval at a data-driven bandwidth, show how the estimate moves when you halve and double h, and never choose the bandwidth that gives the answer you hoped for.

Manipulation of the running variable

Continuity fails if units can sort themselves across the threshold. If some narrow losers could overturn their result, the seats just above the cutoff would include races that were really lost, and those parties may differ systematically from honest narrow winners. McCrary (2008) proposed a check: test whether the density of the running variable jumps at the cutoff. oe.rddensity implements the local polynomial density estimator of Cattaneo, Jansson and Ma (2020); in version 0.3.20a1 you supply its bandwidth yourself.

The block below moves about half of the races lost by less than 2 points to just above the cutoff, then tests the original and the manipulated margins with h = 10.

rng = np.random.default_rng(7)
moved = (df.margin > -2) & (df.margin < 0) & (rng.uniform(size=len(df)) < 0.5)
rigged = pd.DataFrame({"margin": np.where(moved, -df.margin, df.margin)})
print(f"Races moved across the cutoff: {moved.sum()}")
for label, data in [("as simulated", df), ("manipulated", rigged)]:
    test = oe.rddensity(data=data, running="margin", h=10, cutoff=0)["density"]
    robust = test[test.method == "Robust"].iloc[0]
    print(f"{label:>12}: density left {robust.left:.4f}, right {robust.right:.4f}, p = {robust.p_value:.4f}")

Races moved across the cutoff: 18
as simulated: density left 0.0101, right 0.0104, p = 0.9398
 manipulated: density left 0.0033, right 0.0174, p = 0.0003

In the original sample the estimated densities on the two sides are almost equal, and the robust p-value is 0.94. After 18 of the 2,000 races are moved, the estimated density just below the cutoff falls to 0.0033 and the density just above rises to 0.0174; the p-value is 0.0003.

Races per one-point margin bin after 18 narrow losses were moved across the cutoff

The chart shows why a formal test helps. Even away from the cutoff, bins with 20 expected races hold between 13 and 26 by chance, so a gap of this size is hard to judge by eye. A clean density test is reassuring, not proof: manipulation in both directions can leave the density smooth. Also check that predetermined characteristics, such as the vote share at the previous election, do not jump at the cutoff. Running oe.rdrobust with such a covariate as the outcome does this.

The estimate is local

RD answers a narrow question: what is the effect for units at the cutoff? In the simulation, winning raises the next vote share by 5 points at the threshold, but the effect shrinks with the margin, and the true average effect among all simulated winners is 3.48 points. No data near the cutoff can reveal that difference. Extending an RD estimate to units far from the threshold needs extra assumptions, such as an effect that does not vary with the running variable. Likewise, the effect of winning a close race may differ from the effect of holding a safe seat.

What else can go wrong

• Another policy uses the same cutoff. The jump then mixes both effects. Check every rule that applies at the threshold.
• Take-up is imperfect. If crossing the cutoff only changes the probability of treatment, the design is fuzzy. Pass the treatment received with fuzzy= to estimate a local average treatment effect for units whose treatment follows the rule.
• The running variable is coarse. Ages in years or rounded scores leave few distinct values near the cutoff. oe.rdrobust adjusts its bandwidth selection for repeated values (masspoints="adjust"), but no option adds information that the data lack.
• Observations share shocks. If races in the same region move together, cluster the standard errors with covariance="cluster" and cluster=.

A short checklist

1. State the running variable, the cutoff and the assignment rule, and decide whether the design is sharp or fuzzy.
2. Plot the outcome against the running variable. oe.rdplot returns the binned means and polynomial fits of a standard RD plot.
3. Test the density of the running variable with oe.rddensity, and test predetermined covariates for jumps.
4. Estimate a local linear model at a data-driven bandwidth and report the robust bias-corrected interval.
5. Report estimates at several bandwidths, such as half and double the chosen one.
6. Describe the result as the effect at the cutoff.

The regression discontinuity lab applies these steps to a synthetic grant cutoff, and the guide to validating econometric results covers checks that apply to any estimate.

References

• Calonico, S., Cattaneo, M. D. and Titiunik, R. (2014). Robust nonparametric confidence intervals for regression-discontinuity designs. Econometrica, 82(6), 2295–2326. doi:10.3982/ECTA11757
• Cattaneo, M. D., Jansson, M. and Ma, X. (2020). Simple local polynomial density estimators. Journal of the American Statistical Association, 115(531), 1449–1455. doi:10.1080/01621459.2019.1635480
• Imbens, G. W. and Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635. doi:10.1016/j.jeconom.2007.05.001
• Lee, D. S. and Lemieux, T. (2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48(2), 281–355. doi:10.1257/jel.48.2.281
• McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698–714. doi:10.1016/j.jeconom.2007.05.005
• Thistlethwaite, D. L. and Campbell, D. T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51(6), 309–317. doi:10.1037/h0044319