We say this in a resigned tone when we find ourselves in an awkward or difficult situation, or when we have to put up with something less than we expected.
‘The butcher hasn’t delivered the joint ‘ said Mrs Jones, ‘so I’ll have to open a tin of something.’
‘Never mind’, said her husband. ‘Worse things happen at sea.’