Should I take my son out of Nursery?

My little boy is 21 months old.  He was seen by a child psychologist at our local assessment place called rainbow house last month.  From that they had said he is developmentally delayed by 6 months and he is definitely thinking of an autism diagnosis but giving him 6 months to see if there is any further development as he is still younger than 2 and they like children to be at least 2 for a diagnosis.

my struggle just now is I put my son into nursery about 2 months ago now to see if it would help bring on his speech and socialising skills however I’m constantly called to go pick him up early due to him being distressed.  When we arrive at the gates he gets very upset and is screaming when I leave.  He doesn’t eat or drink while he is there either.  They’ve said he settles after 10 mins or so but then gets unsettled again which is when I get phoned.  

they’ve said to keep giving it a chance and to keep going and that it takes some kids longer but in my gut I just feel he’s too young and maybe I should take him out and try again in 6 months when he’s been seen by the psychologist again and also speech and language as he does not say any words at all. I think that’s also what worries me is he can’t tell me what’s upsetting him he can’t communicate with me however his behaviour at home after a nursery session is concerning, he’ll be unsettled and start to hit me and his dad and be very distressed. 

Am I over reacting or should I go with my gut? Any help would be appreciated.