Areas that experienced settler colonialism tended to have one overriding characteristic: a (relatively) low population density, and thus the absence of land-limited systems. That was true in most of Kenya/Tanganyika, South Africa, Australia, much of NZ, and the Americas at the time of initial settler colonization. If land is not available, the lure for new settlers is weak; if the local population is large and well-organized, they can (even in a colony) freeze out new arrivals or simply threaten to create an ungovernable situation, as happened in the Gold Coast Colony several times in the early 1900s relative to schemes to settle white cocoa planters. Weaver's The Great Land Rush and the Making of the Modern World is good about this relative to the Anglo world; the legal and spatial machinery (and its expansion to adjacent zones by force, where those advantages didn't exist so clearly) is remarkably similar. Some cases, like Eritrea and Algeria, not to mention the Portuguese empire during the Estado Novo period and the "adjacencies" like Zimbabwe, are arguably different and relied far more heavily on the application of force and reaped that particular whirlwind during the liberation era. During earlier periods, before the 20th century, tropical Africa and SE Asia also had the liability of ecology; not only did Europeans usually lack good knowledge about making profits except by mining or other capital-intensive (and honestly non-settlement-needing) activities compared to the locals, but the potential threat of endemic disease was something they might avoid.
If you look at the incentives for new settlement in tropical areas after 1900 you can see some of this at work. For example, in order to try and get white settlers into the East Africa Protectorate (and later Tanganyika as well) the colonial governments and the Colonial Office itself had a significant veterans' land grant program, and built a legal regime that favored them. It had only limited success, except in certain sectors of cash-cropping. I can't imagine Europeans doing better with agricultural production in the Mekong Delta than the people already there (under suitable colonial administration, per the French).
In the South African case, it is more temperate, or can be suitably irrigated with boreholes, while most tropical diseases are restricted to the northeastern Lowveld and Limpopo valley. The initial settlement was specifically in an area that could support European crops natively, and only along the Mediterranean coast do you get a similar situation. Europeans were willing to go; the benefits outweighed the liabilities. In the tropical world, the liabilities usually outweighed the benefits for potential settlers unless governments put their fingers on the scale, which they rarely felt was worth the trouble especially if it would upset local people and thus incur monetary costs.
These colonies were set up with different intentions in their respective time periods. People were sent originally from England to America to set up land colonies simply because there were so many problems in England at the time - most of the "colonizers" were very, very poor. When we think of the Jamestown settlement in 1605, known as the first lasting settlement (In the first settlement in the Roanoke Island [present-day North Carolina], the colonizers ran off to live with the Natives), only merchants were sent. This led to increased relation with Native populations, as the settlers relied on the Natives to survive. They settled there and those who came over next were farmers who were displaced by enclosure - land was being used to raise sheep, since wool and textiles were becoming huge economic engines. Puritans later came over to set up religious colonies in New England in the later 17th century and early 18th (Benjamin Franklin was born and raised in a Puritan community).
However, Africa was a different story altogether. The Portuguese discovered African settlements as far back at the 15th century, and used them largely for trade. They had already fought in North Africa, akin to Spanish reconquista (Christians driving the Muslim Umayyad empire out of Spain in the 13th century). They opened up trade ports in Africa and used the ports only - the land was already settled and highly civilized. (Also, the Portuguese discovered slavery here - it was already used amongst themselves as economic fuel.) This kind of colonialism grew in popularity because very little expenditures and diplomacy were necessary, whereas sending settlers to live and take land from Natives was very expensive in money and manpower.
Hope this helps, but ask more if I need to be more specific!
Resistance to disease/displacement is a big factor here, as is economic opportunity. I will try to stay within the realms of my knowledge on white settlement in South Africa, which was sort of a half way stop between a settler colony and the other cases you are asking about. Part of the success (if you can call it that) of European settlement in the Americas and in Australia/New Zealand was that native populations either shrunk drastically or disappeared altogether with more European encroachment. Some of those who wanted British South Africa to become a white settler colony like Australia or Canada also believed that the native populations would shrink there as well and that Britons would pour in to outnumber and eventually outbreed the hostile Dutch-speaking European population nearby. The thing is, though, none of that happened. I believe the native population in South Africa actually increased under British rule/influence, but I don't remember for sure.
British emigrants mostly chose the US as their final destination, but those who stayed within the empire overwhelmingly chose Canada or Australia despite some relatively serious efforts to get people to choose South Africa. The only thing that drove Europeans there was the opening of gold and diamond mines in the 1860s-70s, aka economic opportunity. Even then, there was a ready supply of native labor to work the mines, so it still didn't attract a ton of immigrants, probably less than one million. The short version, I suppose, is that it was impossible settle in a region with an established and healthy population (Asia and Africa), and that native labor was adequate for the natural resources that influenced Europeans to colonize there in the first place. In settler colonies like in North America or Australia, on the other hand, Europeans were moving into areas already thinned out by disease and with no significant labor force, so there was economic opportunity at all points in the spectrum for any Europeans looking to emigrate.
I think you can't underestimate the climate. the sub continent, for example is an incredibly uncomfortable climate for people raised in cold, rainy England. Look at white colonial literature for an idea of that. Burmese days by George Orwell comes to mind as a particular example.
Charles Mann discussed this in 1493 (the sequel to 1491, both excellent).
If I recall correctly he made the distinction between colonies in the malaria zone, which became "extractive" colonies (removal of resources), and those in temperate non-malarial regions which were more likely to be settled.
Simply put because the nature of the indigenous societies didnt make them seem places that colonies were required.
The cultural gap between the Aborigines/NativeAmericans was sufficient that their lands were perceived as empty or emptyish. Especially in Australia where the most advanced culture on Earth met one of the least advanced cultures. People in hunter gatherer communities or early farming ones also have a low population density.
So if the country is good, and there are no nationstates in existance, just a bunch of 'wild men', then it was considered 'may the best man win'.
Indian culture was for example considered quite advanced and there was a lot of opposition to the East India Company's practices. This along with Indias structured society meant that no one ever considered colonising India as it was already occupied.
Africa, central at least, was considered fairly primitive, but heavily occupied, so controlling it, yes, colonise, no. Except of course for the far more temperate and much lower populated south.
It wasnt Europes intention to colonise the world, just those bits they deemed 'emptyish'. I think South/Central America was similar, but somewhat 1/2 way between the Indian and Australian examples, and the Iberians had their own way of doing things.