Anthropic’s Lawyer Apologizes After Claude AI Hallucinates Legal Citation

In a recent legal filing, a lawyer representing Anthropic admitted that they had used a fabricated citation generated by Claude, Anthropic’s AI chatbot, in their ongoing copyright dispute with music publishers. The admission was made in a filing submitted to a Northern California court on Thursday.

According to the filing, which was first reported by Bloomberg, Claude hallucinated the citation, providing “an inaccurate title and inaccurate authors.” Anthropic’s legal team explained that their standard “manual citation check” failed to identify this error, as well as several other inaccuracies caused by Claude’s hallucinations.

Anthropic has since apologized for the mistake, characterizing it as “an honest citation mistake and not a fabrication of authority.”

The revelation follows accusations earlier in the week by lawyers representing Universal Music Group and other music publishers. They alleged that Anthropic’s expert witness, Olivia Chen, had used Claude to cite nonexistent articles in her testimony. Judge Susan van Keulen subsequently ordered Anthropic to respond to these allegations.

The underlying lawsuit is part of a broader conflict between copyright holders and tech companies concerning the use of copyrighted material in the training of generative AI models.

This incident is the latest in a string of cases where legal professionals have faced repercussions for relying on AI-generated content. A California judge recently criticized law firms for submitting “bogus AI-generated research” to the court. Earlier this year, an Australian lawyer was found to have used ChatGPT to prepare court documents containing faulty citations.

Despite these challenges, investment in AI for legal applications continues to grow. Harvey, a startup that uses generative AI to assist lawyers, is reportedly in talks to raise over $250 million at a valuation of $5 billion.