You can apply even before filing of Annual return also. section 54 is applicable in this case. If tax rate of your outward supply is lesser than your Input supply, you can claim refund of Input Tax.
in this case you can claim refund only on rate difference on input. and not on input service or capital goods.
Can you specify the goods you are dealings.... as refund under Inverted duty structure on few goods was block in few goods. Kindly refer Notification 5/2017 CT (R) : specify the goods on which Refund cannot be claimed Under Inverted Duty Structure. And Refer Notification 20/2018 CT (r) : Amending Notification 5/2017 CT(R) allowing refund claim on IDS