I think General Penalty upto Rs. 50,000 under Section 125 of the CGST Act 2017 should be levied.
Filing of FORM GST ITC-04 is prescribed under Section 143 of the CGST Act 2017. No Penalty or Late Fees is separately mentioned for violating this Provision under Section 143 or any other Section.
Thus as per my view "GENERAL PENALTY U/s 125 of the CGST Act 2017" upto Rs. 50,000 ( Rs. 25,000 under CGST + Rs. 25,000 under SGST ) may be levied.
As per notification no.38/2019-central tax ITC-04 upto march-19 is not required to file, give the details of the material not returned up to 1 year and paid the tax on the same.