It's not a hate crime as far as I understand it, but it would appear having read the posts below it would certainly be viewed has harassment. You'll need to keep a record of his actions (days and times) as well as the impact, be realistic in terms of impact, don't exaggerate or it will most likley be ignored. If you've got 2+ weeks of this you can go to the police and register a harassment complaint, though if you can get a full months record you'll find you have much more to push with.
Someone else has said about speaking to your MP but keep that powder dry, go to the police first and only go to your MP if they fail to act. You should make it very clear that you are looking to report harassment and that you know that harassment is a crime otherwise you might find that they attempt to fob you off.