Terrified of work

I'm too exhaused to even type this. I have started a new job and its a completely different career from what I'm used to. Its librarian work which you would think would suit me and sometimes it does but im used to being a teacher and the change to a new career, particularly after how badly I was discriminated against in my last job and the effect that had on me, has absolutley floored me.
Since starting this job I have had so much fear every morning and evening and at the end of lunch breaks about going back. The wierd things are there are also times when i feel very calm there and enjoy the job.

Its difficult to describe the effect its having on me. the only time in my life I went through something similar was the change when I left retail to start teaching. Its very very hard to put into words what is going through my head but it is extremely painful 

Any advice would be needed. I know it might pass when I get used to the job but I dont think I can stand this pain and suffering in the meantime