This is a huge and complex topic which delves into disciplines other than history, but I'll try clear up some misconceptions you have about the world. I'll attempt to answer this question in two parts, first about why the regions you mentioned are so wealthy and why the 'rest of the world' is poorer.

Your question is tied in with two basic ideas in geography - the concept of economic development levels and the Demographic Transition Model.

Nations can roughly be divided into two categories - More Economically Developed Nations (MEDC) and Lesser Economically Developed Nations (LEDC). The division isn't based around their GDP or wealth, but rather the stage they are at in the Demographic Transition Model.

If you look at the chart, you will notice that there are 5 stages. They track a nation's progress through industrialisation. Stage 1 is a nation pre-industrialization, characterized by high birth and death rates and an economy which is mostly primary or subsistence based.

Industrialization is the absolute key. The three nations/areas you mentioned - Europe, the USA and Japan - were the first few parts of the world to industrialize. The technological advancements conferred by industrialization vastly improved the quality of life of the nations undergoing it. Mass production of food, clothing and other goods via the factory system as well as better healthcare in the form of sanitation, fresh water pipes, manufactured medicines, etc. This led to a falling death rate and a rising or consistently high birth rate. (A.K.A. Stage 2)

The details of the latter stages aren't that important to your question, but note that in general the affluence and wealth of a nation increases over time as it progresses through the stages of the Demographic Transition Model. The diversity of economies is also tied to the model, as post-industrial nations undergo the process of 'tertiarization', where increasing amounts of workers change from primary and secondary jobs to the tertiary sector.

Basically, once nations begin the process of industrialization they start on a path that ultimately leads to increased wealth and a diversified economy. A simple explanation is that Western Europe, the USA and Japan are wealthy regions because those nations embarked on the process of industrialization before the rest of the world did, and consquently are further along the Demographic Transition Model.

The second half of the question can be attributed largely to colonialism. The early industrialization of the aforementioned areas came at a vast cost to the other parts of the world. Of course there was direct imperialism in which nations were subjugated and their wealth plundered. India under the British Raj is the textbook example of this.

A massive contributing factor that is often overlooked by some is that local artisans were simply unable to compete against the factory system. In the early 18th century, India was the world’s only major exporter of woven textiles. One estimate places India’s share of the world income at 27% in 1700, compared to Europe’s proportion of 23%. However, by 1870, after nearly a century of European dominance, India’s economic might had fallen to a mere 12% of world GDP. This decline is attributed by many to the trade of mass manufactured British cotton cloth. It proliferated into Indian markets in greater quantities and at lower costs than what local Indian artisan weavers and spinners could offer.

The important thing to note is what replaced the old textile industries - Historian John Darwin asserts that “India’s economy was deliberately shaped to British rule” and points to the primary exports from India in 1870 as an example: wheat, raw cotton, jute and tea. This was commercial agriculture, designed for export to satisfy the cravings of British citizens, not the Indian locals. The resultant effect of this sustained exposure was ‘deindustrialisation’. Due to the collapsing and unprofitable manufacturing sector, the primary sector of India swelled and (European) landowners began growing cash crops for export to Europe. The conquered region regressed in industrial capacity until it fully loses its self-sufficiency and becomes what Hobsbawm calls an “economic planet in the British solar system." J. R. McNeill & William H. McNeill summarize the new British world system with the statement “around the world… the efficiency of British factories also brought distress."

Basically, colonialism left extreme long term damage in certain places around the world, India being the prime example. Historian Rajat Kanta Ray argues that India was forced to rebuild it's industrial capacity from the ground up after decolonisation, a process which is still ongoing today. Areas which weren't directly touched by European imperialism such as Persia and the Ottoman Empire still suffered from the power of the factory system.

I just want to end on the note that your question is a bit out of date. Perhaps 20 years ago your premises would be correct, but the last couple decades has seen the rise of Asia as China and India begin to progress further down the demographic transition model. Just google 'China GDP' or 'India GDP' to see their meteoric rise. Several African nations such as Nigeria have also attained incredible growth rates.

Sources

J. R. McNeill & William H. McNeill, The Human Web: A Bird’s-Eye View of World History, New York, 2003

Angus Maddison, The World Economy, Volume 1-2, Paris, 2006

Angus Maddison, Contours of the World Economy 1-2030AD, Oxford, 2007

John Darwin, The Empire Project: The Rise and Fall of the British World-System 1830–1970, Cambridge, 2009

E. J. Hobsbawm, The Age of Empire: 1875 – 1914, London, 1987