My lack of memory affecting marriage

My dear, if someone keeps a meticulous track record of everything you've ever said or done that was "wrong," considers your legitimate diagnosis as an "excuse" for "poor behavior," makes everything feel like it's your "fault" and that you're the "cause" of his "unhappiness," and you're generally confused and have "no idea of he is correct or not," then this isn't normal, and it makes me a little afraid for you. You might be experiencing "gaslighting" as well as possible other forms of manipulation. I can't diagnose your husband with anything, but he might have some undiagnosed personality disorder. But I can see from the kinds of things he says to you, that he has the personality of a "Taker," and you probably have the personality of a "Giver" (and those are terms I am allowed to use). Just make sure you're not draining all your energy trying to make him happy, and being taken advantage of.

If your husband can support you and your diagnosis and make you feel comfortable and confident with yourself and who you are, then that is fine, but if he is treating you like an inconvenience and a problem to him and his life, and you're draining yourself to try and make him happy, but he is never happy and keeps putting you down and making you feel like you're always wrong and at fault somehow, then I hope you get some help. I can't advise you to leave him (although that's always an option), but if he keeps treating you poorly, then I hope that you'll get help at the very least. But hopefully he's supporting you and he's there when you need him.  

My wife remembers so many things that I can't remember, whereas I can remember random facts! It can be a bit annoying, though it's more proof that brains are different. 

We are all different, no two people present with the same symptoms, just a sliding scale of them. Everyone has to accept those differences and work with them, don't look at your differences as a problem, use them to improve your life, there will be a way you just have to think of a solution.

I understand you have memory problems, so do I, although my memory problems are a defense mechanism I have developed to protect myself. Most people don't listen to everything they hear or try to memorize everything the read or see, Unfortunately that's exactly what I do, I use the word unfortunately because having a memory like that is a curse. I hate being the only person in the room that remembers events that happen 10 or 20 years ago, because I cant mention those events, if I do most people who don't remember will call me a liar, arguments will happen and I will spend years in seclusion. I just have to watch them make the same mistakes again and again.

However I do remember when my memory problems changed, I learned how to control what I remember or rather I learned how to forget, it was a conscious decision on my part after I worked out neuro typical people never focused on one topic in there conversations, so I started using my autistic power and stopped trying to force myself to focus on a subject even if I excelled in that subject. So now I can probably still remember most things I just don't try too. 

Having said all this, we all use autism as both a crutch and an excuse for our behavior. we need to support ourselves when life doesn't quite work out the way we want it. but its never a reason to just give up on trying to understand other people. Yes we sometimes have difficulty expressing ourselves to our friends and family and we deserve their understanding, but they are also having trouble expressing themselves to us as well, and just like us they deserve respect.

Don't just give up because you are autistic, work it out, ask for help if you can, strive to understand them and be more than the person you see in the mirror.