Home › Research › Replication Lab

Replication Lab

Classic studies that forecast Supreme Court decisions, rebuilt from their published methods and run on the cases the Court has decided since OT2022. Each study page profiles the paper, checks our rebuild against the authors' own reported results, then applies the same method to recent terms, scored against the simplest baseline: the petitioner always wins.

Why the baseline matters. The Court reverses the lower court in roughly two of every three argued cases, so a forecaster that always picks the petitioner is right about 65–70% of the time without knowing anything about a case. A method earns its keep only by beating that number on the same cases. Our own forecasts are held to the same test on the scorecard.

Study 1

Ruger, Kim, Martin & Quinn (2004)

The Supreme Court Forecasting Project

Six case characteristics and a set of classification trees beat 83 legal experts at forecasting the 2002 Term.

Paper: 75% (OT2002) · On OT2022–OT2025: 65.8% of 222 cases vs 69.4% for “petitioner always wins”

Study 2

Katz, Bommarito & Blackman (2017)

A general approach for predicting the behavior of the Supreme Court of the United States

One random forest, retrained every term, forecasts two centuries of Supreme Court votes and cases.

Paper: 70.2% (1816–2015) · On OT2022–OT2025: 69.1% of 233 cases vs 66.5% for always predicting reversal

Study 3

Kaufman, Kraft & Sen (2019)

Improving Supreme Court Forecasting Using Boosted Decision Trees

Boosted decision trees that add what the Justices did at oral argument: whom they questioned, how much, and whom they cut off.

Paper: 74.0% (OT2005–2014, cross-validated) · On OT2022–OT2025: 68.6% of 229 cases vs 67.2% for “petitioner always wins”

Study 4

George & Epstein (1992)

On the Nature of Supreme Court Decision Making

Law or politics? A legal model and a political model of the Court's death-penalty decisions, tested head to head.

Paper: 88% combined (1971–88, in sample) · On OT2022–OT2025: 87.5% of 8 cases vs 87.5% for the more common earlier outcome

How each replication works

  1. Profile the study. The question, data, method and findings, from the paper itself.
  2. Rebuild the method from the paper's description, using the Supreme Court Database, and list every place we had to depart from it.
  3. Check the rebuild against the authors' own reported results on their own test cases.
  4. Apply it to OT2022–OT2025. Each term is forecast using only earlier terms, then scored against “the petitioner always wins” on the same cases.

All results are published as JSON under /data/replication/.