What's wrong with the word 'no'

Seriously, why can't NTs just say no, or they don't know. Why so they just ignore you or not respond?? If you don't want to do something, there is a very simple word to use... No. Instead they leave you in limbo not knowing. I'm probably supposed to know that a lack of response means no, but my brain insists I can't say for sure until I get an answer. If this just me?