I'm not a historian, but I have read quite a bit about the history of timekeeping and clocks.

Your latter suggestion is exactly right: each town had its own time. For example, Oxford was about five minutes off from London.

Local time was almost entirely based on the movement of the Sun - that is, the average movement of the Sun. Noon actually fluctuates quite a bit and can occur up to 16 minutes on either side of "average" noon.

Sometimes, especially in larger cities, you wouldn't need to figure out the time yourself. A local master clock could be kept accurate vs. the Sun, while smaller clocks owned by businesses and wealthy townspeople could be synchronized with the larger clock. Bells and chimes were of course a common way of communicating local time over wide areas.

These master clocks were generally installed on churches - monks being the primary late-medieval people who needed to know what time it was - and later on (starting in Germany I believe) government buildings like town halls. This transition isn’t a coincidence; controlling local time is power.

Apart from the need to meet for business purposes - and of course the monks - there wasn't much need for precision. If you said that something would happen "at 8 o'clock", and the master clock read 8:23 when it happened, nobody would bat an eye. The stagecoach schedules in your linked book are almost certainly approximates. Since the mail stagecoach was already the fastest communication method (apart from direct courier), nobody could call ahead and let you know that the stagecoach had broken a wheel and so was an hour behind schedule. It was best to be loose with your expectations where travel was involved.

Two things changed that, both of which you mention in your post: factory work and railways.

For railways, it was mainly the "pain in the ass" factor of publishing a separate schedule on town time for each town in the railway. It was certainly doable, and they did for a while, but eventually the railway's home office typically won out. Times would be published according to each railway's own local noon. But now you had a different problem: each railway had its own head office in its own town, and so you had roughly one time system per railway company.

For factory work, the problem is that your boss's clock is what mattered and he could set it to whatever approximate time he wanted. If you needed to be at work at 9:00 and your watch said it was 8:56, but his said it was 9:01, well, you're fired. The first compromise was to allow neutral third parties - jewelers, as it happened - to set "official" local time, based on synchronization via telegraph and/or railway. Telegraph and railway companies actually offered subscriber services to jewelers to be attached to their time network, boasting about their accuracy vs. other time carriers.

Notice that at this point - we're well into the 1870s by now - we do have synchronized clocks, but still also local time. Local master clocks were electrically kept synchronized with one another, but all that means is that Oxford didn't have to figure out for themselves when "five minutes behind Greenwich" was.

The benefits of standard time zones were obvious to anybody affected by the problem, but there was immense popular resistance to the idea of a standardized time that was not tied to the Sun. Some people fretted that virtually everything else in modern human society had been torn away from natural rhythms; if time could be made artificial, too, then we may as well just be a race of machines. Others fretted that if government edict can overrule the Sun, what other ridiculous things could they force you to do?

Of course, ultimately the "pain in the ass" factor won out and standard time zones were slowly adopted around the world in the period leading up to World War 1. This ran into problems too, mainly for political reasons - France wondered, why should London get to be the "zero time" point and not Paris?! - but ultimately those were resolved in favor of business convenience.

Source: mainly Keeping Watch: A History of American Time with some Time in History thrown in.

Edit; added some detail to the earlier bits