Financial consultant hired to file service tax return failed to do so.

Dear Friends,

I had hired a financial consultant for filing of returns (Income Tax & Service Tax). Few years back he failed to file Service Tax Return for one financial quarter , though the Service Tax was paid by me and I had forwarded the Challan to him in the respective month of that year. Professional Fees was also paid to him for all e-filing services for that year. 

Now I've received a notice to pay a penalty of 20000/- 

Since he was filing returns on my behalf, am I liable to pay the penalty or should I ask him to pay it. 

Please help.