Talking and Tones

Does anyone else struggle with how people perceive their tone? Well today it annoyed me more than usual, as it happened for the first time at my newish job. I love my job and they are understanding but its like sometimes its like they forget I am autistic. I get it, I mask daily and people don't tend to see the 'real' me. I was late diagnosed and have gone my whole life masking and building a different 'me' for things such as work, social situations or even family events.

But even then the mask can drop, being tired, a loud environment and sensory issues can slip through. I was surrounded by 5 people near my desk, having their own conversations while also trying to ask me questions at the same time, this would be a lot for anyone but when someone got annoyed I wasn't listening to them, I replied 'Sorry, I didn't hear you as a lot is going on, people talking and sensory wise', I answered their question and moved on but the energy changed. 

People seem off and now for what ever reason I feel bad, like I should of just put up with it. I am trying to reason with myself to say what I said was nothing rude or wrong but I cant help thinking about it and 'What if I was rude'. 

Does anyone ever get like this? Or does anyone have any insight, I would really appreciate it!