ChatGPT Editing Adding Hallucinations to Otherwise Accurate Material?

A Connecticut state appeals court has ruled that attorney David Gottlieb and his law firm must take corrective actions following allegations that the use of ChatGPT, a generative artificial intelligence tool, introduced incorrect legal citations into court filings. The decision came after a review of the materials submitted in the case Tov Realty LLC v. Suarez, where the court found that while the core legal arguments presented remained accurate, the supporting references had been altered or fabricated by the AI system. According to the court’s ruling, Gottlieb initially conducted thorough legal research using LexisNexis and verified all citations through the Shepard's Citations Service before copying and pasting his work into ChatGPT. His intention was to streamline the drafting process and enhance the clarity of his documents. However, the AI-generated versions of the documents included new or modified case citations, which rendered some of the supporting references inaccurate. These changes went unnoticed during internal reviews by Gottlieb and his colleagues, who focused primarily on the substance of the legal arguments rather than verifying the updated citations. The inaccuracies were brought to the court’s attention by amicus curiae representatives from the Jerome N. Frank Legal Services Organization, who identified what they described as “hallucinated” case citations in the briefs. Upon notification, Gottlieb promptly filed errata sheets to correct the errors in both cases. Nevertheless, the court issued a sua sponte order requiring him and his firm to explain how such errors occurred. In response, Gottlieb and his firm acknowledged their responsibility and expressed regret over the mistakes. They explained that the errors stemmed from the use of ChatGPT to refine the structure and quality of their initial drafts, a step they believed would not alter the factual accuracy of their work. They emphasized that their original research had been verified prior to inputting the content into the AI tool. As part of the sanctions imposed by the court, Gottlieb is required to complete additional continuing legal education beyond the standard annual requirement. Specifically, he must undertake six hours of training focusing on ethics and law office management, with three of those hours dedicated to understanding the implications of generative AI in legal practice. Both Gottlieb and his firm are also mandated to donate $1,000 each to the CT Bar Institute, Inc., to support educational initiatives related to legal ethics, law office management, and the integration of AI technologies in the legal field. The court ordered Gottlieb and his firm to submit a compliance report within six months of the ruling, detailing whether they have fulfilled the outlined requirements. This report must be filed with the Office of the Appellate Clerk and sent as a courtesy copy to the Statewide Grievance Committee. Additionally, Gottlieb must adhere to any reciprocal discipline measures required by jurisdictions where he holds bar admission. The court emphasized that the provision of the compliance report to the Statewide Grievance Committee does not constitute a formal referral for further disciplinary proceedings. The ruling itself will be published on the Judicial Branch website, ensuring transparency and serving as a cautionary example for other legal professionals considering the use of AI tools in their work. The incident highlights growing concerns among legal practitioners about the reliability of AI systems in professional settings. While AI tools can offer valuable assistance in organizing and refining written content, this case underscores the necessity for users to remain vigilant and perform rigorous verification of all citations and references after utilizing such technology. As legal professionals increasingly integrate AI into their workflows, the need for clear guidelines and ethical standards becomes ever more pressing.