Robust and clustered standard errors in Python, from HC0 to the wild bootstrap

OLS coefficients do not depend on how you compute their standard errors, but every test and confidence interval does. This guide explains the common choices, shows them on one dataset with OpenEconometrics, and ends with a case where the choice changes the conclusion.

The sandwich estimator

With classical standard errors, the covariance of the OLS estimator is σ̂²(X′X)⁻¹. That formula assumes every error has the same variance. White (1980) showed that the covariance can be estimated consistently without that assumption, using the squared residuals:

V̂ = (X′X)⁻¹ ( Σᵢ ωᵢ ûᵢ² xᵢ xᵢ′ ) (X′X)⁻¹

The variants differ only in the weight ωᵢ given to each squared residual. Here n is the number of observations, k the number of coefficients and hᵢᵢ the leverage of observation i.

MacKinnon and White (1985) studied finite-sample alternatives to White's estimator, including a jackknife estimator that HC3 closely approximates. In the simulation designs of Long and Ervin (2000), HC0 often led to incorrect inferences with 250 observations or fewer, while HC3 worked well even with 25. They recommended HC3 for samples below 250.

One dataset, several estimates

The example below fits the same regression with each covariance option. The data are the 480 synthetic observations bundled with OpenEconometrics.

import openecon as oe

df = oe.example()
options = {
    "nonrobust": {}, "HC0": {}, "HC1": {}, "HC2": {}, "HC3": {},
    "bootstrap": {"reps": 999, "seed": 123},
    "jackknife": {},
}
for covariance, extra in options.items():
    fit = oe.ols(data=df, y="wage", x=["education", "experience"],
                 covariance=covariance, **extra)
    print(covariance, [round(c.std_error, 4) for c in fit.coefficients])

With openecon 0.3.20a1 this prints the following standard errors:

The estimates are close because these data show little heteroskedasticity. The Breusch–Pagan test (fit.hettest()) gives p = 0.21 and White's test (fit.white_test()) gives p = 0.55. Robust standard errors are not always larger than classical ones. Here the robust intercept error is smaller. The robust estimators protect the inference when the variance is not constant, and they cost little when it is.

The bootstrap result uses normal inference and records its seed and draw algorithm, so the same call reproduces it. OpenEconometrics also offers two refinements for HC2 and HC3: dfadjust=True applies Bell–McCaffrey degrees of freedom to single contrasts, and hansen=True applies Hansen's HC3 adjustment to t statistics and confidence intervals. The OLS documentation describes both.

Clustered errors

When errors are correlated within groups, such as students in a school or firms in an industry, heteroskedasticity-robust standard errors can be misleading because they ignore within-group dependence. The cluster-robust estimator sums the scores within each of the G clusters before taking outer products. Its common CR1 form applies a finite-sample factor:

V̂_CR1 = G/(G − 1) · (N − 1)/(N − K) · (X′X)⁻¹ ( Σ_g X_g′ û_g û_g′ X_g ) (X′X)⁻¹

Abadie, Athey, Imbens and Wooldridge (2023) argue that clustering is a design question. Cluster when the sampling or the assignment of treatment is clustered, not just because the data contain a grouping variable. Cameron and Miller (2015) give a practical guide to the estimator and its assumptions.

In OpenEconometrics, cluster="firm" selects CR1 errors. A list such as cluster=["firm", "year"] gives multiway clustering with up to four dimensions. Inference uses a t distribution with G − 1 degrees of freedom, taking the smallest G across dimensions for multiway clustering. For one-way clustering, covariance="cluster_hc3" adds a leverage correction within clusters.

When there are only a few clusters

Cluster-robust inference relies on the number of clusters being large. With few clusters, t tests based on CR1 can reject far too often (Cameron, Gelbach and Miller 2008; MacKinnon, Nielsen and Webb 2023). The wild cluster bootstrap with the null hypothesis imposed is the standard remedy.

The example data has a region column with only four values. The regions are synthetic and not a real clustering design, but they show what happens when G = 4:

fit = oe.ols(data=df, y="wage", x=["experience", "education"],
             covariance="cluster", cluster="region")

webb = oe.wild_cluster_test(fit, data=df, null={"experience": 0},
                            cluster="region", wild="webb", reps=9999, seed=1)
signs = oe.wild_cluster_test(fit, data=df, null={"experience": 0},
                             cluster="region", wild="rademacher",
                             enumerate_all=True)

The three procedures disagree by orders of magnitude. With four clusters there are only 2⁴ = 16 Rademacher sign patterns, and two of them reproduce the original statistic, so the p-value can never fall below 2/16 = 0.125. Webb's six-point distribution was designed for this situation: with four clusters it allows 6⁴ = 1,296 distinct patterns. None of these numbers makes four clusters a sound design. The honest conclusion is that the data cannot support precise cluster-robust inference.

Serial correlation: HAC errors

For time series, Newey and West (1987) proposed a heteroskedasticity and autocorrelation consistent (HAC) estimator that stays positive semi-definite. It weights autocovariances of the scores with a kernel up to a chosen lag. In OpenEconometrics, use covariance="hac" with time="period", lags=4 and a kernel such as kernel="bartlett", or call oe.newey(...). With lags="auto", the lag length follows the plug-in rule of Newey and West (1994) for the Bartlett, Parzen and quadratic-spectral kernels.

A short checklist

• Decide what to cluster on from the sampling or assignment design before looking at the results.
• Report the covariance estimator, the number of clusters and the degrees of freedom. OpenEconometrics writes them into its table notes.
• Prefer HC3 to HC0 or HC1 in small samples.
• When clusters are few or very unequal in size, check the result with a wild cluster bootstrap.
• Treat a large gap between procedures as information about the design, not as an invitation to pick the smallest p-value.

References

• Abadie, A., Athey, S., Imbens, G. W. and Wooldridge, J. M. (2023). When should you adjust standard errors for clustering? Quarterly Journal of Economics, 138(1), 1–35. doi:10.1093/qje/qjac038
• Cameron, A. C., Gelbach, J. B. and Miller, D. L. (2008). Bootstrap-based improvements for inference with clustered errors. Review of Economics and Statistics, 90(3), 414–427. doi:10.1162/rest.90.3.414
• Cameron, A. C. and Miller, D. L. (2015). A practitioner's guide to cluster-robust inference. Journal of Human Resources, 50(2), 317–372. doi:10.3368/jhr.50.2.317
• Long, J. S. and Ervin, L. H. (2000). Using heteroscedasticity consistent standard errors in the linear regression model. The American Statistician, 54(3), 217–224. doi:10.1080/00031305.2000.10474549
• MacKinnon, J. G., Nielsen, M. Ø. and Webb, M. D. (2023). Cluster-robust inference: A guide to empirical practice. Journal of Econometrics, 232(2), 272–299. doi:10.1016/j.jeconom.2022.04.001
• MacKinnon, J. G. and White, H. (1985). Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties. Journal of Econometrics, 29(3), 305–325. doi:10.1016/0304-4076(85)90158-7
• Newey, W. K. and West, K. D. (1987). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55(3), 703–708. doi:10.2307/1913610
• Newey, W. K. and West, K. D. (1994). Automatic lag selection in covariance matrix estimation. Review of Economic Studies, 61(4), 631–653. doi:10.2307/2297912
• White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48(4), 817–838. doi:10.2307/1912934