With one’s tail between one’s legs

Appearing frightened or cowardly (like a frightened dog).

The man left the store with his tail between his legs when someone saw him trying to steal something.