Struggling to live with partner

My partner has moved in with me and my daughter. Prior to this, the only person I had ever lived with/shared a home with was my family growing up and then my daughter when she was born (I was a single mum from the get go). 

I remember before he moved in I was quite apprehensive how I would cope because I am someone who *needs* my own space/alone time and even though I have my daughter, I managed this when she was at school or staying over at relative's homes etc. I had a good routine with housework, my home was always very clean and tidy, everything had its place etc and I was able to pursue my special interests etc. 

I've really struggled with him living with us for various reasons. Because he is home quite a lot due to working from home and working shifts ie he is not in work everyday I feel I don't have enough time to keep to my routines and pursue special interests. I can't fully be myself when he's home i.e. I cannot stim freely. He has a lot of his own stuff, which I really struggled with at the beginning, my home is quite small and there isn't enough space for all our stuff. I really don't like it when he moves my things or tries to change things around. Now the flat is usually quite messy and cluttered which I hate. I also find that the cleaning routine I had before he moved in has gone out the window. For some reason I don't like cleaning/tidying when he is at home I find it distracting. I feel when he is at home I can't do anything and I also find it frustrating that everything just gets messy very quickly. 

I also find him quite loud, like he has a very loud voice or will be singing in the kitchen and I am very sensitive to noise particularly if I'm stressed or overloaded. He is also the sort of person who will have the tv on all the time as background noise, even if no one is watching it which I don't like. I don't mind a bit of good music but it really annoys me having some random show on when no one is watching. 

I feel like our relationship is quite strained because I find him being here quite frustrating and stressful and a lot of it is probably ASD related although no all of it (sometimes I feel like we don't have a lot in common, don't have particularly interesting conversations etc)

I've decided that I will end the relationship and ask him to move out. I feel bad because he is otherwise a good person. He keeps saying he loves me and I feel he is close to my daughter. But I don't think I can carry on much longer. I long to have my flat to myself and miss the days when it was just me and my daughter (I've never struggled when it was just her and me, I think mother's love overrides a lot of difficulties). 

Has anyone else struggled when a partner has moved in? I can't see me living with anyone else again tbh. I clearly need my own space.