India’s Supreme Court and the Fruit of the AI Hallucinated Tree

Hallucinated outputs by Artificial Intelligence (AI) models are the kind of phenomenon that, if foretold forcefully enough, might well have caused India’s judiciary to question the wisdom of opening its doors to the usage of AI at all.

That time, though, has passed. Unsurprisingly, questions regarding that wisdom are now pressing with some urgency.

Following efforts over the past year by High Courts at Kerala and Gujarat to set out their AI use policies, India’s Supreme Court released a set of draft regulations for the use of AI in Indian courts this past June. The Supreme Court’s top line is ambitious. It gives its blessing to the use of AI in Indian courts for “legal research, precedent retrieval, citation verification, and document summarization.” It says, however, that AI use is to “remain strictly subservient to human judgment and judicial authority.”

However, sixteen documented instances of AI hallucinations in active cases across the country — including six before the Supreme Court itself in the past ten months alone — have called a lie on the ‘human judgment’ safeguard that the Supreme Court envisioned.

The Supreme Court’s response to this trend in recent months has been aggressively anti-AI.

But how far can — and should — it go?

According to Damien Charlotin’s excellent database, there are sixteen documented instances of judicial proceedings in India whose records have been compromised by AI hallucinations, including six before the Supreme Court of India in the last 10 months alone (Photo by Arvind Yadav/Hindustan Times via Getty Images).