Hold (something) against (someone)

To blame something on someone.

My friend forgot to give me my money but I do not hold it against him as he is a nice person.