The Pony Express (April 1860-October 1861) was made obsolete more than it failed as a business model. It was perfect for the moment, but the moment quickly passed as soon as the West Coast was linked with “the states” with the telegraph.
It is a great point that you make about the Pony Express having such a lasting legacy in the nation’s perception of the West. It was a short-lived institution, and yet it is remembered: re-enactment groups conduct annual Pony Express rides, and for its centennial, one could have mail send (and stamps cancelled) by Pony Express.
Why the romanticism? It is directly tied to the perception of the Wild West – which lasted no longer than the Pony Express, if the Wild West existed at all. The Pony Express captures the romantic image of the West: vast, rugged, dependent on horses, fast, thrilling, and dangerous. Young men (orphans preferred said the advertisements for riders) rushing across the continent, dodging all sorts of dangers, and making the nation connected at a time when the nation was threatened with division – it doesn’t get better than that. And all of this happened on horses racing as fast as they could: who doesn’t like a fast horse?. This is all practically the definition of the Wild West.
Did the Wild West exist? Not really, but if there was a moment when it seemed to come to life, it was in those months when the Pony Express rider risked it all.
Because it was intended as a publicity stunt to win back the transcontinental mail contract, and while it didn't do the latter, the stunt part clearly worked.
So let's start by stepping back a bit. The Post Office was the largest part of the federal government for its first few decades which dwarfed all other aspects; it hovered around 70% of federal employees until the late 1840s. Even John C. Calhoun wouldn't touch the continuous expansion of subsidized postal routes; Members of Congress could go to a desk, pull out a bill in progress, and write on it their addition to the long list of routes and post offices that would be authorized and funded in a pro forma vote at the end of the session.
This meant that mail contracts subsidizing the expansion of stagecoach service went hand in hand for the first 50 years or so of the United States. Richard John in his comprehensive Spreading the News has done analysis suggesting that some rural routes - especially in the South - got subsidies up to 70% from the federal government. There's an interesting parallel today in that while Acela in the Northeast is about the only profitable part of Amtrak, the only stagecoach routes that didn't need subsidies were the Northeast.
Congress realizes this and cuts significant amounts of subsidies in 1845 and begins sending some mail on the burgeoning rail lines, but in rural locations the gravy train continues. Enter John Butterfield, a self made man who'd started out as a stage coach driver on those lucrative Northeastern routes, and did well enough so that he founded several transportation companies of his own (including, with others, American Express in 1850.)
When in 1857 several Western members of Congress push for more regular mail service to the West Coast and pass legislation that spring, Butterfield bids on multiple options of it; one from St. Louis to San Francisco as the legislation requires, another from Memphis to Los Angeles, and a third that's brilliant from a political standpoint: it goes from both Memphis and St. Louis to satisfy South and North. He also sets a provision in this third bid to allow the Postmaster General to determine the best route, which includes sending both sets of coaches to an undetermined point (what would become El Paso, Texas, wins, with just a minor bit of pushing by of one of Texas' senators) before it goes on to San Francisco and Los Angeles.
With this politically sensitive creativity, in September Butterfield wins the new mail service contract at the maximum reimbursement rate of $600,000 per year and has his hands full; the legislation requires him to commence service within one year, which he beats by 3 days. This is equivalent to about $20 million today, so the contract isn't lucrative by itself but a tremendous boost if you're going to run a for-profit passenger and goods stagecoach line. John suggests it accounted for about 30% of Overland Mail's revenue, and this number doesn't include the kicker of 320 acre land grants at 10 mile intervals along the route for waystations; Butterfield chose 20 mile intervals and apparently held onto the rest.
Butterfield was no slouch in terms of publicity himself. A 1942 book that is still used by writers as a basic reference on the Overland Mail route - The Butterfield Overland Mail by Waterman Lily Ormsby - is mostly composed of the 6 lengthy dispatches he sent back to the New York Herald along the way describing his trip as the first ever through passenger on the route from St. Louis to San Francisco from September 26th to November 19, 1858. Ormsby was an intrepid 23 year old reporter for the Herald, but his nominal employer didn't pay the fare; Butterfield did so himself - and traveled a good part of the trip with Ormsby.
As a result, it's little surprise that among the topics sent back in the serials was a defense of why the Postmaster General had selected this particular route and Butterfield in particular to run it. Like everything else touching on sectionalism in the 1850s, the route itself was very controversial; it was originally supposed to be at the 35th parallel, but remember that bid term allowing the Postmaster General to choose the route? It comes into play since Aaron Brown (of Tennessee) selects a radically different one running South along the 32nd parallel. This routing was constantly criticized, although in some small irony Butterfield tended to hire drivers and stationkeepers he'd employed previously - who generally hailed from New York. For that matter, Ormsby also took it on the chin; he was skewered by competing newspapers as a shill and employee of Butterfield, but his reporting appears to be fairly legitimate - even if he took a steamer back to New York.
So at the start the Overland mail route took, give or take, about 24 days over its 2700 miles; it's required by the legislation to be done in a maximum of 25, and by 1859 and 1860 it drops to just under 22 (probably helped by a brutal schedule that allows for no more than 10 minute stops at waystations.) One way fares run $100 as an intro deal, then $200, then settle at $150 - with top-of-the-line coaches on parts of the route fitting 9 people under normal circumstances, something like 15 or so if both drivers benches are used, and even more than that if they stuff people onto the roof (which they apparently did, although they switched to lighter coaches with less capacity through the far West.)
So if you want to wrest the contract away from Butterfield, how are you going to do so? You can't compete much on price - Butterfield eventually gets booted from his own Board since the Overland is barely breaking even - you can make noises about the route safety but don't have much to back it up (there's only a single attack in its three years of running in Arizona with no fatalities, although when passing through Apache and Comanche lands drivers usually arm themselves even if Butterfield doesn't require it), so what's left? Speed.
And that's where William Russell and the Pony Express step in. Russell had run a couple of reasonably profitable stagecoach companies out West, but when the mail subsidies got taken away Russell started losing a good deal of money. His one real hope was trying to revive the 35th parallel route and regain not just the previous mail subsidies but wrest away the bigger contract from Butterfield.
I'll let John's analysis stand on its own here since it explains plenty.
"But this does not mean that the postal system played no role in Russell's plans. On the contrary, Russell intended from the outset to use the Pony Express as a publicity stunt to convince Congress and the postmaster general to give him the stagecoach contract for the Overland Mail. To beat out the competition, Russell later reminisced, he had tried to build himself a "world-wide reputation"; the Pony Express was a means to that end. Pony Express rider J. H. Keetley put it rather more bluntly. From start to finish, Keetley later maintained, the venture was nothing more than a "put-up job" for the Overland Mail. To maximize publicity, he recalled, Russell required the first rider that left St. Joseph wear an outlandish costume that included silver trappings, a scabbard, and jingling spurs, making him resemble a "fantastic circus rider." Once the rider reached the boat to cross the Missouri River, he quickly packed the costume away, so that it could be used by the next rider making the trip."
Russell lost money hand over fist, but did indeed capture the nation's attention with cutting the time of the trip in half with some serious panache. Unfortunately for him, it wasn't just the telegraph lines being completed that put him out of business in 1861; with the Civil War, there was no safe land mail route available, and Overland Mail shut down too.
I'll defer to /u/itsallfolklore on how Bill Cody and others kept part of the mythos alive, but as a business venture it did exactly what it was intended - just not the part that was supposed to get the mail contract to keep it solvent!