We use this idiom to describe the relief we feel after we tell somebody a secret which is troubling us or something important which is difficult to share.
After cheating on his wife, Ben was feeling bad about it, so he got it off his chest and told her everything. Of course, she divorced him.