I've just come across a bag of stones I've kept for years. I have a thing for stones I dint know why but I can forget time if I'm looking at or for unusual stones.
I also have gardening as a hobby that I think about way too much and spend hours on but I think that's a more socially acceptable hobby in a nt world
I like collecting stones as well, and fossils. I've been collecting them since I was a child, and would happily spend hours at a time searching the beach on holidays. I don't know much about geology, though.
Most are quite common and ordinary, I know that I could buy much better ones in a shop or online, but I'm not bothered. I did buy myself a chunk of fossilised wood once, but I don't have the same feeling of attachment to it. I enjoy searching, and looking at what I've found reminds me of happy times.
There's also that moment, (which David Attenborough describes in one of his programmes), where you split a rock and find yourself looking at a fossil which hasn't seen the light of day for millions of years, and which you are the first human to see. It is, as he describes, a "moment of magic."
I do this too! I'd spend 3 or 4 hours walking and forget the time as I was just searching for stones fosils or seaglass. I've bought a few gem stones but I like searching more like yourself