Have a rough time (of it)

To experience a difficult period.

My friend had a rough time of it when he was forced to leave his job.