Michael Cohen supplies fake AI legal citations to lawyer 

Occurred: November-December 2023

ChatGPT generated false legal citations that were used in a legal filing involving Donald Trump's former lawyer Michael Cohen, highlighting concerns about the system's propensity to 'hallucinate' 'facts' and users' poor understanding of its risks.

In November 2023, Michael Cohen submitted legal citations he believed to be accurate to his lawyer David Schwartz that he thought might help his request for an early end to court supervision. Cohen had pled guilty to tax evasion and campaign finance violations in 2018 and served time in prison. 

However, a court filing (pdf) revealed the citations had been generated by Google's Bard chatbot, and were false - an admission made after the judge overseeing Cohen’s case had said that he could not find any of the decisions Schwartz had cited and demanded an explanation. 

In a sworn statement, Cohen said: 'As a non-lawyer, I have not kept up with the emerging trends (and related risks) in legal technology and did not realize that Google Bard was a generative text service that, like Chat-GPT, could show citations and descriptions that looked real but actually were not'.

Operator: Michael Cohen
Developer: Microsoft
Country: USA
Sector: Business/professional services
Purpose: Generate text
Technology: Chatbot; NLP/text analysis; Neural network; Deep learning; Machine learning; Reinforcement learning
Issue: Accuracy/reliability
Transparency: 

Page info
Type: Incident
Published: January 2024