Time rigidity & routine stress

Hi! I'm newly diagnosed (3 days ago!) so I'm doing loads of research to try to understand myself better and work out in what ways autism affects me and if others can relate. I've always struggled with other people choosing a time that we do things in the day, this is because I usually have a time that I want to suggest, but feel too awkward to suggest it, then they suggest it and it's not the 'right' time, so it messes up the plan I had already made in my head and often also worries me with things like food and how exhausted I'll be after seeing someone.

For example; 

My grandma had asked me to go to her house to discuss my autism diagnosis (she is supportive and interested in knowing more), so I said okay and that I'd go today. I'd rather not, but she values time spent together a lot. I woke up this morning at 7am and personally would have liked to of gone at 8 or 9, to get it out the way, but I decided in my head that 10am would be okay because then I could be home for lunch and before 12.. after 12 is the afternoon and that means half the day is gone, that's too late. She suggested 11am, a time I HATE but that NT's seem to love for social things. 11 stresses me out so much because I'll be hungry for lunch by 12, but i'll be expected to stay at hers until 1, so I'll end up coming home at 1/1:30, hungry, feeling unwell, tired from socialising and then have to somehow cope with the rest of my day. I said 'okay', because I'm too anxious to ask her if she can do earlier. I feel angry that she suggested 11 and upset with how this affects the rest of my day.

I don't know if this is time rigidity, needing to be in control, routine, food preferences or all of the above, but does anyone else relate? I hate that I go through this cycle every time I have to see someone. I just want to get things out of the way early, so I can go home and follow my preferred schedule.