Cards are stacked against (someone)

Luck is against someone.

The cards have been stacked against the young boy since he was born.