What happened

During a hearing on July 19, 2024, a Melbourne-based lawyer submitted a list of purported case precedents that neither the judge nor her staff could verify. 

Upon investigation, it was revealed that the citations had been fabricated by LEAP's LawY AI tool and that the lawyer had not confirmed their accuracy before presenting them in court, leading to a postponement of the hearing and a slap on the wrist by the legal authorities.

Why it happened

The lawyer admitted to using Leap's AI capabilities without understanding it's hallucinatory limitations. 

He ultimately issued an apology and offered to cover costs incurred by the other party due to the adjournment.

What it means

The incident raises concerns about the accuracy and reliability of LEAP's generative AI software, and highlights the importance of verifying AI-generated information. 

It is part of a broader trend in which multiple lawyers have faced repercussions for relying on unreliable generative AI systems without properly checking their outputs and, as such, raises questions about the ethical implications of using AI in legal practice.

Similar cases have emerged globally, including instances in Canada and the UK where legal professionals cited fabricated cases generated by tools like ChatGPT.