OpenEconometrics

Replication 09 · Quarterly Journal of Economics, 1999

Maimonides' Rule and Class Size

How much do smaller classes raise the reading and math scores of Israeli fourth and fifth graders?

Joshua D. Angrist, Victor Lavy · Quarterly Journal of Economics 114(2), 533–575 (1999) · Instrumental variables · fuzzy regression discontinuity · two-stage least squares · first stage and reduced form · clustered standard errors

Replication 09 video lesson: Maimonides' Rule and Class SizeWatch the lesson · 20:22

In 1991, Israeli fifth-grade classes with more pupils had higher reading scores, not lower. Does that mean large classes help? Angrist and Lavy found a cleaner comparison in a rule that goes back to the twelfth-century scholar Maimonides: no class may have more than 40 pupils, so a grade of 40 gets one class and a grade of 41 gets two. This lab reruns Tables I to V of the paper from the authors' public data and Stata programs in OpenEconometrics, compares all 602 printed numbers with the run, and explains every difference.

Do smaller classes raise test scores?

The authors had only class averages, not scores for individual pupils, so every observation is a class: its average reading and math scores on a 0–100 scale, its size, the enrollment in its grade and the percent of disadvantaged pupils in its school. The fifth-grade sample has 2,019 classes in 1,002 schools; the fourth-grade sample has 2,049 classes in 1,013 schools. An average fifth-grade class has 29.9 pupils in a grade of 77.7, with mean scores of 74.4 in reading and 67.3 in math. On average 14.1% of a school's pupils are disadvantaged, and such schools receive extra resources.

Table II for fifth graders, from the Replication 09 run: OLS slopes of class-average scores on class size, with controls added in turn, and 95% intervals.

Without controls, ordinary least squares says that one more pupil goes with 0.221 more reading points and 0.322 more math points. Adding the percent of disadvantaged pupils turns the reading slope to −0.031 and cuts the math slope to 0.076; adding enrollment as well leaves both close to zero (−0.025 and 0.019). Disadvantaged pupils are more often in small classes and score lower, so OLS mixes the effect of class size with the effect of who is in the class.

Maimonides' rule

Since 1969, Israeli public schools have split a grade into classes of at most 40 pupils. With enrollment e in the grade, the rule predicts an average class size f_sc = e / (int((e − 1) / 40) + 1). A grade of 40 has one class of 40; a grade of 41 has two classes of 20.5; 81 pupils make three classes of 27, and 121 pupils four classes of 30.25. Predicted class size rises with enrollment and falls sharply at 41, 81 and 121.

The paper's Figure I for fifth graders, rebuilt from the authors' data: Maimonides' rule and the average actual class size at each enrollment.

Actual class size follows the sawtooth, with softer teeth. In schools with 36 to 40 fifth graders, classes average 29.7 pupils, where the rule predicts 38.0; with 41 to 45 they average 23.0, where the rule predicts 21.4. Class size falls by 6.7 pupils where the rule predicts 16.6, and the raw drops at the higher cutoffs are smaller still. The rule is followed only in part, so it cannot replace class size, but it can serve as an instrument for it: a fuzzy regression discontinuity.

Two conditions make it work. Relevance: f_sc must move actual class size, which the first stage measures. Exclusion: f_sc must affect scores only through class size, so that a school with 41 pupils in the grade is like one with 40 except for its smaller classes. Because f_sc is built from enrollment, and larger schools tend to have better-off pupils, every specification also controls for enrollment: linearly, as a quadratic, or with a piecewise linear trend. Then only the jumps of the rule identify the effect of class size.

First stage, reduced form and 2SLS

Regressing class size on f_sc and the percent disadvantaged gives a first-stage slope of 0.704 (Table III, column 1); with enrollment added it is 0.542, both as printed. Each pupil of predicted class size adds about half a pupil of actual class size. The reduced form regresses reading scores directly on f_sc: with both controls the slope is −0.150, where the paper prints −.149; the gap comes from one class in the data, described in the scorecard.

With one instrument, two-stage least squares is the ratio of the two slopes: −0.1500 ÷ 0.5415 = −0.2770, the 2SLS estimate in Table IV, column 2. The run checks this identity to machine precision in four columns.

Table IV, readingReduced formFirst stageRatio2SLS
Column 1 · % disadvantaged−0.11150.7035−0.1585−0.1585
Column 2 · + enrollment−0.15000.5415−0.2770−0.2770
Column 5 · discontinuity sample−0.19740.4812−0.4102−0.4102
Column 6 · discontinuity + enrollment−0.20170.3463−0.5824−0.5824
Indirect least squares in the Replication 09 run: the reduced form divided by the first stage equals the 2SLS estimate in each column.

The main result

Table IV, fifth graders, from the Replication 09 run: 2SLS estimates of the effect of one more pupil on class-average scores, with 95% intervals from cluster-robust standard errors.

With the percent disadvantaged as the only control, one more pupil lowers class-average reading by 0.158 points. With enrollment added, the estimate is −0.277 (paper −.275); with a quadratic in enrollment it is −0.263, and with the piecewise linear trend −0.190. For math with both controls it is −0.231 (paper −.230); without the enrollment control it is close to zero (−0.013). The sign is the opposite of OLS: bigger classes lower scores.

Cutting a fifth-grade class by ten pupils raises reading by about 2.8 points, 0.36 standard deviations of class averages, and math by about 2.3 points, 0.24 standard deviations. The paper's own example, 8 pupils × .275 = 2.2 points or an effect size of .29, and its per-pupil effect sizes of .036 and .071 all reproduce. These standard deviations are of class averages; in terms of pupils' scores the effect would look smaller, and these data cannot measure it.

Near the cutoffs and in fourth grade

The discontinuity sample keeps only schools within five pupils of a cutoff, enrollments 36–45, 76–85 and 116–125: 471 fifth-grade classes in 224 schools, and 415 fourth-grade classes in 195 schools. There the 2SLS estimates for reading are −0.410 with the percent disadvantaged and −0.582 with enrollment added, both as printed: larger than in the full sample, but less precise. The cluster-robust standard error of column 6 is 0.206, against 0.076 in column 2. Enrollment stays in the model so that the comparison rests on the jump at each cutoff, not on the slope within each window.

2SLS, class size5th gradePaper4th gradePaper
Reading · full sample (col. 2)−0.277 (0.076)−.275 (.066)−0.133 (0.061)−.133 (.059)
Math · full sample (col. 8)−0.231 (0.099)−.230 (.092)−0.050 (0.075)−.050 (.070)
Reading · discontinuity sample (col. 6)−0.582 (0.206)−.582 (.181)−0.150 (0.132)−.150 (.128)
Math · discontinuity sample (col. 12)−0.443 (0.251)−.443 (.236)0.023 (0.159).023 (.160)
2SLS estimates for fifth and fourth graders from the Replication 09 run, with cluster-robust standard errors, next to the printed values and their Moulton standard errors.

Table V repeats everything for fourth graders. The first stage is as strong (0.772, as printed), but the reading effect is about half as large, −0.133 per pupil, and the math effect, −0.050, cannot be told apart from zero. A ten-pupil cut would raise fourth-grade reading by about 1.3 points. The replication reproduces this contrast; it cannot say why the older pupils gain more.

Two modern checks, computed with the same data and school clusters, are extensions rather than numbers printed in the paper. The Montiel Olea–Pflueger effective F statistic is 217.9 for the fifth-grade first stage and 415.5 for fourth graders, against a critical value of 23.1 for a 10% worst-case bias. The cluster-robust Anderson–Rubin 95% confidence set for fifth-grade reading is [−0.434, −0.134], close to the Wald interval [−0.426, −0.128]; for fourth-grade math it is [−0.199, 0.096] and includes zero.

The scorecard

TargetPaperThis labResult
Table I · 5th-grade classes and schools2019 · 10022019 · 1002✓ Match
Table III · first stage, f_sc (col. 1).7040.7035✓ Match
Table III · reduced form, reading (col. 4)−.149−0.1500Difference
Table IV · 2SLS, reading (col. 2)−.275−0.2770Difference
Table IV · col. 2, imputed class at its school mean (diagnostic)−.275−0.2754✓ Match
Table IV · 2SLS, math (col. 8)−.230−0.2311Difference
Table IV · 2SLS, reading, discontinuity sample (col. 6)−.582−0.5824✓ Match
Table V · 2SLS, reading, 4th grade (col. 2)−.133−0.1329✓ Match
Table V · 2SLS, math, 4th grade (col. 8)−.050−0.0497✓ Match
Table IV · col. 2 SE, cluster-robust (SDK).0660.0759Difference
Table IV · col. 2 SE, Moulton translation (diagnostic).0660.0660✓ Match
Text, pp. 567–568 · four effect sizes2.2 · .29 · .036 · .0714 of 4 match✓ Match
All printed SEs · Moulton translation (diagnostic)148144 of 148Difference
Tables I–V · every printed number602428 match · 30 rounding · 144 differDifference
The paper's headline numbers next to the Replication 09 run, with the two labelled diagnostics, and the count for Tables I to V.
TableNumbersMatchRoundingDiffer
Table I · Descriptive statistics13413310
Table II · OLS8450727
Table III · Reduced forms2041401450
Table IV · 2SLS, 5th grade9042741
Table V · 2SLS, 4th grade9063126
Tables I–V60242830144
Every number printed in Tables I to V, compared with the run at the printed precision, table by table.

Of the 602 numbers printed in Tables I to V, 428 match at the printed precision and 30 more match under a named rounding convention. 144 differ, and every one has a diagnosis; none is unexplained. Every sample size and every count of classes and schools matches, and Table I matches 133 of its 134 numbers, the last one by rounding. The differences fall into three groups.

Standard errors (106). The paper corrects its standard errors with Moulton's (1986) formula, which assumes one random shock per school, shared by all its classes. OpenEconometrics has no Moulton estimator, so the scorecard uses its cluster-robust standard errors by school, which allow any correlation within a school and are larger here: 0.0759 for Table IV, column 2, against .066 printed. As a labelled diagnostic, the lab applies the inflation factor of the authors' mmoulton_post.do to the SDK's conventional standard error: 0.0551 × 1.198 = 0.0660. Done this way, 139 of the 148 printed standard errors match and 5 more round, 144 in all; the other 4 differ by one unit in the third decimal. No point estimate depends on this.

Four standard errors for Table IV, column 2: the SDK's conventional and cluster-robust ones, the Moulton diagnostic built on the SDK fit, and the printed value.

One imputed class (32). One fifth-grade class (school 21194, 5 pupils tested) has stored scores of 187.6 in reading and 181.2 in math, impossible on a 0–100 scale. They equal the school's total, 63 pupils averaging 73, minus its other classes, divided by the 5 pupils tested. The authors' programs subtract 100, which gives 87.6. A diagnostic refit with the class at its school's mean marks, 73 in reading and 69 in math, stored in the same file, reproduces all 32 differing numbers, and a search over scores from 55 to 95 in half-point steps finds that only these two values reproduce all 36 printed fifth-grade numbers of each subject. The headline moves from −0.277 to −0.2754, which rounds to the printed −.275. The scorecard keeps the authors' file. The class lies outside the discontinuity windows, so those estimates are unaffected.

Slips in the paper (6). Table II prints fifth-grade standard deviations of 8.1 and 9.9 where Tables I, III and IV print 7.7 and 9.6 for the same classes (its reading mean of 74.3 against 74.4 is a rounding difference). It prints 69.9 for the fourth-grade math mean, where Tables I and III print 68.9. Table V prints 67.3 (9.6), the fifth-grade math mean and standard deviation of Table IV, instead of the fourth-grade 68.9 (8.8). And Table V, column 8, prints −.020 for the enrollment coefficient; the run gives +0.0198, and every other fourth-grade math enrollment coefficient is positive, so the sign was most likely lost. Table I's heading also gives the last window as 116–124, while the programs, and the printed counts of 471 and 415 classes, use 116–125.

Where this replication stops

The lab reruns Tables I to V with OpenEconometrics estimators. Stata was not run, and the deposit contains no logs, so every target is a published number. The third-grade rows of Table I and Tables VI to VIII are not reproduced: the deposit has no 1992 data, and those tables are outside this lab's scope. Figures II and III are not redrawn. Moulton standard errors are reported only as a diagnostic, never as an SDK result.

The replication checks the paper's numbers, not its design. The data are class averages for Israeli fourth and fifth graders in 1991. Whether enrollment near a cutoff was pushed, or anything else changes at 40, 80 and 120 pupils, is a question this lab does not test; later research has revisited Israeli class sizes with that concern in mind.

Run it yourself

Watch the companion video above. The lab package will be published here. The authors' files are public domain (CC0) on Harvard Dataverse:

  1. Run prepare_sources.py from the lab package. It downloads the 8 files the lab uses from the authors' Dataverse deposit, about 0.5 MB in all, and checks each one against its recorded SHA-256 fingerprint. The same data and programs are on Joshua Angrist's data archive at MIT.
  2. Open a Python document in OpenEconometrics, point the launcher at the lab, the authors' files and a new results folder, and press Run.
from lab import run_lab

result = run_lab(
    source_root=source_dir,
    output_dir=results_dir,
    display_callback=display,
)

Each 2SLS estimate is one call:

fit = oe.ivregress(data=iv, y="avgverb", x=["tipuach", "c_size"], endog=["classize"], instruments=["func1"], cluster="schlcode", small=True)

The run writes reconciliation.csv, with all 602 printed numbers next to the lab's results and a diagnosis for every difference.

The paper

Joshua D. Angrist and Victor Lavy, "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement", Quarterly Journal of Economics 114(2), 533–575, 1999. doi:10.1162/003355399556061. Data and programs: doi:10.7910/DVN/XRSUJU, CC0 1.0. Joshua Angrist shared the 2021 Nobel prize in economics with Guido Imbens and David Card.

Try it on your own data

OpenEconometrics is free and open source under Apache 2.0. Install the Python library or download the desktop alpha.

Download Desktop App pip install openecon