Logo Hotsuto
All Newsletters Latest Emails Categories
Pricing What's New Log In Sign Up

Marginal REVOLUTION

Claude 3 for SCOTUS?

06-17-2024

I would vote to confirm: I decided to do a little more empirical testing of AI’s legal ability. Specifically, I downloaded the briefs in every Supreme Court merits case that has been decided so far this Term, inputted them into Claude 3 Opus (the best…
Read on blog or Reader
Site logo image Marginal REVOLUTION Read on blog or Reader

Claude 3 for SCOTUS?

Tyler Cowen

June 17

I would vote to confirm:

I decided to do a little more empirical testing of AI’s legal ability. Specifically, I downloaded the briefs in every Supreme Court merits case that has been decided so far this Term, inputted them into Claude 3 Opus (the best version of Claude), and then asked a few follow-up questions. (Although I used Claude for this exercise, one would likely get similar results with GPT-4.)

The results were otherworldly. Claude is fully capable of acting as a Supreme Court Justice right now. When used as a law clerk, Claude is easily as insightful and accurate as human clerks, while towering over humans in efficiency.

Let’s start with the easiest thing I asked Claude to do: adjudicate Supreme Court cases. Claude consistently decides cases correctly. When it gets the case “wrong”—meaning, decides it differently from how the Supreme Court decided it—its disposition is invariably reasonable...

Of the 37 merits cases decided so far this Term,1 Claude decided 27 in the same way the Supreme Court did.2 In the other 10 (such as Campos-Chaves), I frequently was more persuaded by Claude’s analysis than the Supreme Court’s. A few of the cases Claude got “wrong” were not Claude’s fault, such as DeVillier v. Texas, in which the Court issued a narrow remand without deciding the question presented.

Although I’ve heard concerns that AI would be “woke,” Claude is studiously moderate.

Here is much more from Adam Unikovsky.  A lot of people are still in denial, or not far enough along to even count as "denying."

Comment

Marginal REVOLUTION © 2024. Manage your email settings or unsubscribe.

WordPress.com and Jetpack Logos

Get the Jetpack app

Subscribe, bookmark, and get real-time notifications - all from one app!

Download Jetpack on Google Play Download Jetpack from the App Store
WordPress.com Logo and Wordmark title=

Automattic, Inc. - 60 29th St. #343, San Francisco, CA 94110  

Respond To Email
Mark
© 2025 hotsuto.com
  • About Us
  • Privacy Policy
  • Terms of Service