Visiting friends is a nightmare

My six-year-old son was diagnosed as on the spectrum last year. Recently his general behaviour has got worse and worse. I'm doing my best but have had old ladies tutting at us in the street and other parents at school (who aren't aware of the issues) looking at me as though I'm the worst parent and he's the worst child in the world. Even friends say the right things but then treat my son as though he is just naughty, making me feel frustrated, angry and upset. Just today I took my son and his two younger brothers to one of his school friends house (teacher training day) and I came away wanting to cry. He gets very angry if he can't do exactly what he wants to do - in this case, play lego. which had been put away before our visit - and will share his frustrations with anyone. He kept telling his friend's mum - who is also supposed to be my friend and is aware he is on the spectrum - that all their toys were boring and girly etc etc. She kept telling me she didn't mind but her tone with him was far from friendly. This is just one of many incidents and now just feel like socialising like this is just too stressful and upsetting for all of us. It certainly isn't the fun time that you'd like it to be. Has anyone else felt like this and has anyone got any suggestions please?