Getting tested / Just a label

I asked my doctor about getting tested for aspergers, and he said as an adult i could only do it privately, and it would be just a label which wont change anything.

Is that right?

I'm 48. Struggled all my life with depression, anxiety, social situations.  Been in the same job 30 years, probably because I can't face job interviews.  And now that my job security is not looking so good, my anxiety is through the roof.

My long term partner, who works with children thinks I have aspergers.  Reading up about it, it fits me to a tee.