Turning down parties

How do you guys go about turning down parties? I can't deal with big social events at all and avoid them like the plague for a variety of reasons (anxious amongst lots of people, will not dress up for going out, hardly like any food so meals are a problem, noise, can't deal with all the conversations). I don't really like to explain this to people especially as I'm not diagnosed so I can't even give that as a reason. I usually go with distance or money. There is a party being organised for work at the moment and several people are trying to persuade me to go. I find these conversations very difficult and they keep coming up with solutions to my reasons for not going. How would other people deal with this?