Service Won't Assess Me

I posted before about a doctor who, while agreeing to refer me, was very dismissive and didn't think I was autistic. It was clear through what he said that he is very uneducated on autism in adults and particularly in women and girls (kept insisting that my teachers would have noticed and got me diagnosed as a child). 

He referred me anyway, and I was sent a Screening Tool Checklist to return, of which I ticked 9/10 choices (only one I left was sensory issues as they are rare for me). and I elaborated where I could in the tiny spaces it left me. 

However, I've received my letter from my county's Adult Asperger Service (I don't know why they have a specific one, or why I was referred there when I asked for a general autism assessment referral) and they have said that they can Only assess for ASC where symptoms cause clinically significant impairment in social, occupational, or other important areas of current functioning. And based off of the information they received about me from my GP (who to add on to his lack of knowledge has met me twice in my life and all since September this year) they don't think I fit the criteria and won't offer me an assessment. They also stated how they only offer assessments to those with the most difficulties because of how many referrals they've gotten so I don't understand why my doctor referred me straight to there. 

I'm just incredibly frustrated that they've based my need for assessment purely on the words of an uneducated doctor, and have made no attempt at an initial appointment with me to see for themselves whether I need assessing. 

Is it worth going back to my doctors to see someone else and ask for a referral and to a different service? (I will see a woman this time as I don't like my previous doctor after how he spoke to me when I asked for the referral).