Pre-locomotive railroads (usually called "wagonways") were largely, if not entirely, restricted to the mining industry. First, they were use in the mines, for minecars carrying ore or coal. These would be pushed by the miners; pushing them means that rails are useful for keeping them going in a straight line. Eventually, rails were installed outside the mine, usually to move the ore/coal to a dock, where it could then be loaded onto boats/barges for water transport. Rails were useful, since they allowed a horse to pull about 4 times as much. The main disadvantage of rails - the vehicles can only go where the rails go - doesn't matter when the whole point is to move stuff from point A (the mine) to point B (the docks). This limitation of being stuck to the rails results from the flanged wheels used with rails performing rather poorly off the rails. There is also the problem that if the car/wagon is loaded with 4 times as much weight as the horse can pull off-rail, the horse can't pull the car/wagon off-rail.

On some wagonways, because they ran downhill from mine to dock, gravity was enough for the journey one-way, with a brake-man stopping the cars from going too fast. The return journey, for the empty cars, would be by horse.

Early rails were wooden, and the longest wooden-rail wagonways were a few kilometres long. Wooden rails were being replaced by wrought iron or cast iron rails before the locomotive. Wrought/cast iron rails were more durable, more efficient, and definitely preferred once iron was cheap enough. This drop in price was a product of the industrial revolution. Wagonways grew longer - up to at least about 8km. The range of goods being carried grew larger, but wagonways still served industrial purposes, moving heavy goods from point A to point B.

Why not more wagonways? They were expensive. Where a sufficient amount of heavy goods were being moved, building them was worth the expense. Where there wasn't, they weren't built. Steam locomotives changed the equation - goods could be moved faster, over longer distances, at lower cost. So, after the locomotive, more railways were built.

As for wheel-rut systems, there's no improvement in rolling resistance between having wheel running in ruts along a flat stone surface and wheels running along a flat stone surface without ruts. Ruts also accumulate dirt which increases the rolling resistance. Since ruts don't help (other than providing automatic steering, which is useful if you are pushing the load rather than pulling it), why would you make ruts? It's possible that roads with ruts that have been interpreted as rut-wagonways are just roads with ruts worn into them. Roads like this (with or without ruts) were expensive to build, and encounter the same cost-effectiveness problem as railways with animal traction to move cars/wagons. Wear and tear to roads can be reduced by using lighter loads. This has the advantage that travel along the roads is easier for the horse (or other animal providing traction), and the load is not too heavy if it is moved off-road. Where traffic will be heavy enough, good roads can be built and maintained. Where traffic will be lighter, vehicles can move off-road. If you load cars/wagons more heavily, the vehicles can't move off-road - loads will have to be moved to off-road wagons or pack animals for off-road transport. Maintenance will cost more, since heavier loads will cause more wear-and-tear on the roads. Construction will cost more, since gradients must be more gentle (or heavy loads can go out of control going downhill, and the extra effort to move them uphill will be beyond the capabilities of the animal or team that can move it on level or almost-level road).

So, in summary, building good roads for regular wagon/cart/car traffic where it will be heavy enough to justify the expense, and letting those same vehicles move off-road otherwise, is a good solution. Animal-drawn rail transport becomes cost effective if heavy enough traffic will be moved from point A to point B. It works best with iron or steel rails, and these only became cheap enough with the industrial revolution. Steam locomotives make rail transport faster and cheaper, and make building railways more attractive; they also displaced horse-drawn rail transport. Thus, the heyday of animal-drawn rail transport was the industrial revolution until the use of steam locomotives - the 18th century and early 19th century was their peak.

References:

Bulliet, Richard. The Wheel : Inventions and Reinventions, Columbia University Press, 2016

Nicholas Crane, The Making Of The British Landscape, Hachette UK, 2016