What happened

In a civil case against insurer State Farm, lawyers from two firms — one using Google Gemini and Westlaw’s AI tools —created an outline for a supplemental brief. 

The draft contained fake legal research and was handed off to another major law firm, K&L Gates, which incorporated the fabricated citations into their filing without verifying their authenticity. 

A judicial review revealed that at least two cited cases did not exist, and nine out of 27 citations were incorrect in some way. 

The judge warned that these errors could have ended up in an official court order, potentially undermining the integrity of the judicial process and eroding trust in legal proceedings.

Why it happened

The fracas resulted from the attorneys’ uncritical reliance on generative AI tools for legal research and drafting, coupled with a lack of proper verification and oversight, with neither firm conducting adequate checks to confirm the authenticity of the AI-generated citations before submitting the brief. 

The judge described this as professionally reckless and emphasised that no competent attorney should outsource legal research and writing to AI without careful review.