Hi there! I'm not a historian, but I have an interest in history and I love railroading, so...

To answer your question in short, the U.S. rail system is only underdeveloped in terms of passenger rail. According to the International Union of Railways, the American rail network is the longest in the world, longer than the next two national systems combined. The reason why it's perceived as having "fallen by the wayside," as you put it, is because American passenger rail is a mere one percent of the world total (UIC 2020). Why did this happen?

First, the American railroad network, unlike almost all other nations, is entirely privately owned and operated, meaning that, in true American fashion, it's operated for-profit. Even Amtrak, the government-funded solution to America's passenger rail problem is operated for profit, though all of its stock is owned by the government. (This leads to its own issues, as Amtrak owns very little of the track it runs on, meaning it pays fees to operate over the track of freight hauling railroads that would rather just run freight...but that's another story.) These railroad companies have all adopted the position--Amtrak included--that passenger trains don't make money, a position that has its roots in the 1960s.

Shortly after World War II, railroads were doing great. They were instrumental to transforming America's economy to meet wartime needs, and had been heavily featured in wartime propaganda as crucial to the war effort, as I'm sure you're aware. In the 1950s, however, that began to change. A double-whammy, the National Interstate and Defense Highway Act of 1956 and the Boeing 707's first commercial flight in 1958, as well as America's rapidly-growing car culture, made rail travel seem obsolete. Railroads were something that your parents rode, something old-fashioned, something out of the 19th Century that had no place in the 20th. Also, railroaders were heavily unionized--which is not necessarily a bad thing, but led to work agreements that required railroad companies to pay five people to run a train when two or three were sufficient (I can’t find a source for this right now, but consider: diesel locomotives, requiring only a single person to operate, replaced steam locomotives, all of which require at least two people to operate, by 1960. Most trains still had two people in the locomotive cab in the mid-1980s, when cabooses were abolished. Paying 3 extra people to sit in an engine or a caboose and essentially do nothing was costing railroads a lot of money. There are also union agreements on pay, which is a whole other thing, and far too long to squeeze into this parenthetical.)

The final blow to passenger rail in the U.S. was that all these for-profit railroad companies were competing with each other. Some railroads had competing mainlines mere feet apart, and in many cases fought for passenger traffic between the same two cities that now were connected by an Interstate highway and new, federally-subsidized airports. Thus, railroads began to merge to cut costs, which in many cases were passenger trains and stations (and track). By the late 1960s, rail travel was at an all-time low, so the Federal Government stepped in and created Amtrak in 1971—a system which was not intended to really succeed, only to be a stop-gap until [passenger trains in the U.S. died a slow death] (https://www.resilience.org/stories/2010-09-19/passenger-trains-our-hope-more-sustainable-future/). Unfortunately for its detractors, people still liked the trains, especially during the 1970s oil crisis, and thus Amtrak has hobbled along to this day, maintaining the bare minimum of connectivity. Various plans have been put in place to improve it, but none have really done the job yet.

Hope that answers your question!

It’s difficult to support the claim that American rail network is underdeveloped. It’s almost as large as that of the entirety of Europe, and has shed (as redundant or unprofitable) nearly half the trackage it once had. It’s just that the US has chosen to keep its network under private ownership, and in recent decades has not chosen to subsidize a new passenger rail network.

Railways originated as private enterprises in the United Kingdom and, shortly thereafter, in the US. As infrastructure undertakings of a scale not previously known, they were among the first enterprises to need a new legal concept: that of the private corporation. The industry, like many breathtaking new inventions, attracted a lot of “irrational exuberance” as well as outright fraud in the organization of the business enterprises and the construction of the lines. Soon, legislators reacted with securities regulation, and many states also forbade public subsidy to railroad companies who—like tech companies today—would pit towns against each other to see which would subsidize most highly a route through their city.

In Europe (and India and Japan), the decades of railway building coincided with the 19th century period of creating modern nation-states, and so railways in financial trouble were more readily brought under state or royal control. In some European nations this happened almost immediately, while in others it occurred in the wake of World War I. The US (and Canada) also built a number of railways to tie together the nations, notably the first Transcontinental Railroad (completed in 1869), but primarily did so by offering grants of free land rather than direct subsidy or taking ownership. Even though American railroads were receiving various kinds of government aid, US politics and public opinion steadfastly treated them as private enterprises, and a brief period of government control during the First World War was always seen an a temporary exigency to be ended as soon as possible.

Geography also played a role. Europe, India, and Japan are places with great density of settlements, meaning not only megacities could easily be connected by rail, but hundreds of smaller cities as well. Only a few regions of the US have similar settlement patterns. On the other hand, North America has vast seas of emptiness to transport coal, grain, petroleum, chemicals, and finished goods across at low cost. Thus, North American railroading came to serve the needs of freight shippers, and of railroad shareholders. Unprofitable passenger operations were abandoned as soon as regulators would permit. (We’re approaching the 50th anniversary of the largest of those transformations, the 1971 creation of Amtrak, on May 1.) In this sense, the US is not terribly different from the nations of South America, South Africa, or Australia. In the US, about 35% of freight moves by rail; in the EU only about 10% does.

For complex interconnected reasons of politics and national pride, a few nations have in the last 50 years chosen to make significant government investment in passenger service, usually entirely new networks of high-speed rail lines. This era began with Japan’s 1966 opening of the Tokyo-Osaka Shinkansen; gained new impetus with 1980s/1990s decisions in France, Spain, and Germany; and spurred smaller networks in Scandinavia, Italy, Southeast Asia, South Korea, and even North Africa. All now have been completely dwarfed by the construction of an enormous new network in China. Political calculations in the US, however, have so far not led to such investment in a national high-speed network. From the postwar era until recently, intercity highways in the US—though not all local roads and streets—were financed entirely by highway users, making a huge subsidy for passenger trains difficult to defend. Energy and environmental considerations that arose in the 1970s just happened to be followed immediately by deregulation of airlines in the US (1978), which led to much intercity demand being met by airlines that received little visible subsidy. Additionally, American land-use patterns don’t support city-center to city-center trips by rail very well. All this reduces the number of city-pairs in the US (and many other nations) where a robust intercity rail market would exist, with short trip demand going to private autos and inexpensive bus service, and very long trips only really practical by air.