Can anyone help me out on this.

Excess TDS paid can be adjusted against future liabilities. At the year end, the excess would appear as a debit balance under current assets. You may adjust this while making payment for the transactions pertaining to April 2021.

Is the nature of excess TDS similar to tax credits?

Yes, you may treat so.

Means, can I bring those April month invoice in the year 2020-21 itself and show it in my books? 

1) If such expense pertains to FY20-21 you may record as part of the year 20-21 or if provision can be made for the same during FY20-21, you may create provisional entry and deduct tax too.

2) Alternatively, if such expense is not related to FY20-21, then it can be accounted only in FY21-22, in which case, adjust excess TDS in the subsequent period while making payment as well as while filing TDS return.

Revise tds return of FY 19-20 Q4 and remove the tds entry from  the tds return. The tds challan will get untilized. 

Use the untilised challan in q1 FY20-21 tds return.

.

Find out any possibility of making provision to a/c the same so that it can be avoided the spill over.