Help

My neighbour is a young autistic adult, he comes round sometimes to help out and earn some pocket money doing gardening and stuff. He gets on my my partner, but I find him so difficult, he talks constantly, he never listens, he'll ask a question and then tell you want you were going to say before you've got the first three words out. I can' thave a conversation with him. His Mother is clinically barking mad, he's started coming round again as she's away at the moment, she's weird around me. I feel unwelcome in my own space around him and nobody seems to get why. I really try and get on with him and I know there's a lot of things he can't really help, but he dosen't try either.

What do I do?