It seems like the technology required (horse drawn vehicles, metal tracks embedded in roads) has been around since Roman times, if not earlier.
It has to do with the advancement in iron casting for wheels and rolled iron for rails. It came to be economically viable thanks to advanvements of the early industrial revolution. Also the ability to machine the wheel profiles using lathes which of course are also a contemporary product of the early 19th century.
"Time of the Trolley" by William D Middleton 1967, kalmbach publishing.