The Delhi High Court has issued a stern warning to the GST and Income Tax departments, advising them to use extreme caution when employing AI tools like ChatGPT for legal citations in show cause notices and assessment proceedings. The court highlighted the risk of 'AI hallucinations' and fabricated case laws, emphasising that AI-generated data is not yet reliable enough to substitute human judgment in legal matters. Departments are now responsible for independently verifying all cited judgments to ensure accuracy and prevent the use of fake legal references.
The Delhi High Court has cautioned both the GST Department and the Income Tax Department to exercise "utmost caution" when relying on Artificial Intelligence (AI) tools such as ChatGPT for legal citations in show cause notices (SCNs) and assessment proceedings.
The warning came from a division bench comprising Justice Prathibha M. Singh and Justice Shail Singh, while hearing a petition seeking to quash a GST show cause notice. Although the court declined to interfere with the SCN and directed t
The Delhi High Court has warned the GST and Income Tax departments to exercise utmost caution when using AI tools like ChatGPT for legal citations in show cause notices and assessment proceedings.
The warning is due to the risk of 'AI hallucinations' and fabricated case laws, where AI tools might generate inaccurate or imaginary legal references without proper verification.
No, the court stated that AI chatbots cannot be relied upon to determine legal or factual issues and should not be used for citing authoritative legal precedent. They may only be used for preliminary understanding or research.
The GST and Income Tax departments must take full responsibility if they cite or generate information using AI software, and all judgments used in official documents must be independently verified.
Potential issues include incorrect interpretations, fabricated case laws, imaginary or inconsistent data, and responses influenced by training data or user queries, highlighting that AI accuracy is still in a 'grey area'.
Finance news reporter covering taxation, GST, income tax, business compliance, and economy updates. I simplify complex financial topics into easy-to-understand articles for professionals, taxpayers, and business owners on leading finance and tax platforms.