Hang back

To stay some distance behind someone or away from someone, to hesitate or be unwilling to do something.

The man always hangs back when his boss asks for volunteers.