Am I being paranoid?

I'm in an office with a man who has mental problems. He can be very kind and very funny, but he hates being told what to do. He's started clicking his pen which drives me insane. I hate being like this but I can't stop it upsetting me. I've told him it's annoying, but he carries on. On Friday I was particularly sensitive as it had been such a stressful week and I was frazzled. I asked what that noise was, as I didn't want to say directly that it was annoying me as that annoys him. He said that it was him. I did say that's annoying. He KNOWS it annoys me! He didn't stop. Then he got up to get something off the printer and while he was waiting for the print he had his pen with him, which isn't usual, and carried on clicking. 

Also, he sometimes puts music on his phone, even though he knows I hate it. It's down low but there's that awful scratchy sound that makes my head tighten up. He wears earphones to play his music so I don't know why he does this.

I've had a lot of support at work, to help me with my disability, so I'm worried people think it's all about me. It's not. I just find the world so overwhelming and being with this man, and the job being intense, and my manager micro-managing, I'm on tenterhooks all the time.

I'm mean, jeez, I'm 55 years old and I feel like a 5 -year-old!

But am I being paranoid that my colleague is deliberately winding me up, being disrespectful and resentful?