Late fee in GSTR 10

Assessee failed to file GSTR 10 within due date after cancellation of registration. Now while filing this GSTR, it is showing a late fee of Rs. 10,000. He also received a notice from department asking him to file GSTR 10. He otherwise do not have any GST liability and is a service provider. Is there a way to avoid this late fee.