Get a rise out of (someone)

To provoke a response from someone.

We got a rise out of the teacher when we opened the window in the cold weather.