Kagan Says She Was Impressed by AI Bot Claude’s Legal Analysis

US Supreme Court Justice Elena Kagan found AI chatbot Claude to have conducted an excellent analysis of a complicated Constitutional dispute.

Kagan, speaking at the Ninth Circuit’s judicial conference in Monterey, Calif., said she has been following a blog by Supreme Court litigator Adam Unikowsky of Jenner & Block LLP, who has undertaken a number of experiments with AI and legal writing. In one blog last year, he asked the chatbot to analyze the high court’s divided opinions involving the Confrontation Clause, where Kagan had authored both majority and dissenting opinions.

“Claude, I thought, did an exceptional job of figuring out an extremely difficult Confrontation Clause issue, one which the court has divided on twice,” Kagan said.

Unikowsky this month published a post where he fed Anthropic PBC’s flagship Claude all of the briefs for a case he had argued last fall and asked the model to act as an attorney providing oral argument to the high court. He concluded that the bot provided better argument than he had.

“On the one hand I thought that was refreshingly humble,” Kagan said of Unikowsky. “But then on the other hand it just seems ridiculous that Claude could do an argument better than he could, or Claude could do it, more to the point, better than I could. I kind of think I’m better than Claude.”

Her comments come as the legal industry attempts to navigate a number of complicated questions about how the profession can adapt to the quickly moving technology—one Chief Justice John Roberts highlighted in his 2023 end-of-year judicial report.

Although Kagan said she was impressed by the model’s abilities, she doesn’t “have the foggiest idea” of how AI will alter the legal industry.

Her remarks also come a day after a federal district judge in New Jersey withdrew a legal opinion that attorneys noted has contained numerous errors including made-up quotes and incorrect case results, mistakes that AI chatbots are known to make. There was no mention of AI in the complaints the attorneys made to the judge.

A growing list of lawyers across state and federal courts have faced reprimand for submitting briefs containing AI-generated citations or other convincing “hallucinations.” A number of courts, including the Ninth Circuit, have set up committees to study the issue.