Have the devil to pay

To have a great deal of trouble.

I will have the devil to pay if I do not return home early.