The "modern" US desire for overseas bases was largely an outgrowth of its postwar WWII security policies. The exchange in essence was that the host countries would not only receive some money for the base (and training and cooperation and some other things) but also some prestige and the safety of the American "nuclear umbrella." In other words, bases abroad was, in most cases, seen as a way of extending US security to other countries who weren't otherwise capable or willing to field a military force that would be able to go up against another superpower, like the USSR.
(Not all US bases fit this criteria, e.g., obviously Guantanamo does not, and it is older than the Cold War. There were overseas bases for earlier periods, but that is a somewhat different dynamic.)
As for why other countries don't have bases in the US, what would the US get from such an "exchange"? The situation is inherently asymmetric — the US would not gain much from having, say, a Dutch military base on its soil, but it gains much (and arguably the Netherlands gain something) from the reverse situation.
For a nice history of the US overseas basing situation, see David Vine, Base Nation.
Late to this party.
/u/restricteddata mentioned it, but a lot of this was borne out of the post-WW2 security policies.
In fact, in General Marshall's Biennial Reports of the Chief of Staff of the United States Army to the Secretary of War: 1 July 1939 - 30 June 1945 (official army.mil link), Marshall writes on 1 September 1945, the topic of future defense/security policy:
The German armies swept over Europe at the very moment we sought to avoid war by assuring ourselves that there could be no war. The security of the United States of America was saved by sea distances, by Allies, and by the errors of a prepared enemy. For probably the last time in the history of warfare those ocean distances were a vital factor in our defense. We may elect again to depend on others and thee whim and error of potential enemies, but if we do we will be carrying the treasure and freedom of this great Nation in a paper bag.
He then writes:
Even as late as 1940 I was asked very much the same question before a committee of Congress. Not even then could I say definitely exactly where we might have to fight, but I did recall that in past wars the United States forces had fought in Latin America, in France, in Belgium, in Germany, in Russia, in Siberia, in Africa, in the Philippines, and in China, but I did not anticipate that in the near future American soldiers would fight in the heat of Burma and in the islands of the vast Pacific, and would be garrisoning areas across the entire land and water masses of the earth. From this lesson there is no alternative but that this Nation must be prepared to defend its interest against any nation or combination of nations which might sometime feel powerful enough to attempt the settlement of political arguments or gain resources or territory by force of arms.
And then:
At the close of the German war in Europe they were just on the outer fringes of the range of fire from an enemy in Europe. Goering stated after his capture that it was a certainty the eastern American cities would have been under rocket bombardment had Germany remained undefeated for two more years. The first attacks would have started much sooner. The technique of war has brought the United States, its homes and factories into the front line of world conflict. They escaped destructive bombardment in the second World War. They would not in a third.
It no longer appears practical to continue what we once conceived as hemispheric defense as a satisfactory basis for our security. We are now concerned with the peace of the entire world. And the peace can only be maintained by the strong.
Emphasis mine.
This is taken from page 209-211 in the section For The Common Defense. As you can see, military thinkers - before Japan even signed the Instrument of Surrender - were already thinking about a new world after WW2 and the necessity for the US to go beyond its isolationist roots and think about defending its interests overseas, far from its shores across the oceans, which were soon to be no longer a viable defense.
I also feel it is important to point out that the bulk of US troops overseas actually are rooted in history as well: the list of US troops stationed overseas, as illustrated in this graphic by Time shows that for individual nations:
Now, Afghanistan is obviously a war zone. But then you see that 3 of the top 4 are the vanquished Axis foes of WW2, all of whom were occupied after the war (and some, like Germany, didn't have their full sovereignty restored until the end of the Cold War).
The other one is South Korea, where US troops have been present since the end of WW2, first to occupy south of the 38th Parallel (with the Soviet Union occupying north) after Japan's defeat, and then as part of the US forces of the UN Command to defend South Korea in the Korean War, and then as part of the Mutual Defense Treaty Between the United States and Republic of Korea signed October 1, 1953.
Finally, I want to point out that while no other countries have military bases in the US - there is a considerable presence of foreign troops in the US that most do not realize.
For instance, here is the US State Department's Foreign Military Training Joint Report to Congress for 2015-2016: https://www.state.gov/documents/organization/265163.pdf
This is an annual report with all unclassified data published online for public use. This file lists each of the 80+ nations that the US had training operations conducted with, including a breakdown by location in the US where it was conducted.
This list excludes NATO nations, Japan, Australia, and South Korea.
In addition, the US has had long term foreign troops stationed in the US.
The German Air Force has been in Holloman Air Force Base, New Mexico (official af.mil link) since 1992. The German Air Force has been in the US since 1958.
The US Air Force also has the Euro-NATO Joint Jet Pilot Training (ENJJPT) program at Sheppard AFB which has been producing jet pilots for the US and 13 other partner nations since 1981.
The Italian Navy sends all of its pilots to the US for training by the US Navy and US Marine Corps. Embassy link here. I would know too - I trained with quite a few of them.
How little do people even realize these foreign troops in the US?
The US Air Force's 428th Fighter Squadron is in fact a Republic of Singapore Air Force unit which has been in the US since 2009. That unit flies the F-15SG.
Yes, tiny Singapore has an Air Force squadron in the US. Correction, it has more than one: the 425th Fighter Squadron is also a Singapore Air Force squadron which has been at Luke AFB, AZ since 1993. They fly the Singaporean version of the F-16.
So while these units are renting spaces in existing US bases that still have US commanders, this is quite akin to the arrangement the US has with may other countries like Soto Cano Air Base, which is a Honduran air base where the US has a detachment - Joint Task Force Bravo located.