The increasing use of Artificial Intelligence (AI) in legal and financial research is raising concerns about its reliability, as seen in a recent tax ruling by the Bengaluru bench of the Income Tax Appellate Tribunal (ITAT). The tribunal passed an order in Buckeye Trust vs PCIT-1 Bangalore (ITA No.1051/Bang/2024) in December 2024 but later discovered that it was based on fictitious Supreme Court and High Court judgments, allegedly sourced from ChatGPT.
AI-Generated Errors Lead to Order Recall
The case revolved around the taxability of a Rs 669 crore trust settlement involving a transfer of interest in a partnership firm. The ITAT ruled in favor of the tax department, citing past Supreme Court and High Court judgments to justify taxation. However, a subsequent review revealed that three Supreme Court and one Madras High Court rulings referenced in the order did not exist.
Sources informed that ChatGPT was used to generate the case law citations, which turned out to be fictitious. Upon this discovery, the ITAT revoked its order within a week, issuing a recall notice on January 7, 2025, citing "inadvertent errors" and scheduling a fresh hearing.
Reliability of AI in Legal Research Questioned
The use of AI tools like ChatGPT in legal research is not uncommon, but this incident underscores its risks. Chartered accountants and tax professionals caution against relying entirely on AI-generated legal references without thorough verification.
While the source of the error remains unclear-whether from the tax department's representative (DR) or the tribunal bench itself-this case highlights the growing challenges of integrating AI in the legal domain.
The ITAT has not officially commented on whether the recall was directly due to the fake case laws. However, this incident serves as a stark reminder that while AI can assist in research, human due diligence remains indispensable in legal and tax rulings.
CAclubindia
