IS ITC-04 FILING Mandatory for merchant exporters to claim REFUND OF ITC ON EXPORTS?

We are merchant exporters with registered office in J&K. we get our goods manufactured in different states in our own label and then exporting directly from the premises of manufacturer . we purchase raw materials locally ( near the site of manufacturing ) and get it delivered at the premises of manufacturer and in the process pay 18% IGST . Shipping bills & all export related documents are in our name . Since the goods we export are zero rated (HSN 230990 ) , we claim refund of ITC paid on input materials . Local Tax authorities are withholding our GST for two years now , primary reason cited is that we have to file ITC-04 . Now i want to understand that as per law , are merchant exporters (traders) supposed to file ITC-04? and if yes , is it mandatory for refund of ITC on exports ? Please note that we have already listed the invoices issued by the registered manufacturers ( for Job work ) under purchases in our monthly 3B filings .