Pain in the neck/ass/butt

An annoying or bothersome thing or person.

Dealing with my neighbor is always a pain in the neck.