Settlers.

No, really. That's the single most useful distinction to start with. White settlers got legislatures, constitutions, and fundamental legal rights. Non-settlers, variously 'natives,' 'indigenes,' 'aborigines,' or any of the other epithets used to describe an existing population, did not.

Furthermore, both could and did exist in the same place. If one looks at the treatment of non-Europeans within settler colonies like Canada (unified mostly in 1867), the colonies of Australia (unified in 1901) and New Zealand, you can see a tiered, hierarchical, dictatorial, 'othering' mode of rule and management used in India or non-settler Africa at work there too. Likewise, if you look at settler colonial rule in Africa, especially South Africa, you see that for the settlers many of the same kinds of local self-rule and legislative structures existed as did elsewhere. People of European, and particularly Western European, origin were not subject to the rather extractive rule by fiat (and violence) that others were, but when they were present in numbers they also demanded and obtained a formal voice in their interests.

In the 19th century, this distinction between groups intersected with ideas of race and civilization in virulent ways, to the point that settler populations were on the whole more brutal to non-'whites' in their midst than were imperial authorities. Such settlers obtained preferences, legal rights, and entitlements that were denied or wrested from existing populations. Colonial holdings like the East Africa Protectorate (Kenya particularly), when settlement schemes put enough Europeans on the ground, could also convert into settler-dominated entities. As in Cape Town, or Livingstone (N. Rhodesia), or anywhere else, devolving 'responsible government' or other elements of self-rule was both a demand from settlers and a way to lift the burden of rule off of the imperial treasury and its services.

At the same time, for Europeans in other non-settler British colonies, they still had privileges and legal rights / expectations that not all possessed; the controversy over Indian judges in the 1880s (the Ilbert Bill issue of 1884), whereby enemies of liberalization of political rights for Indians raised the specter of Indians judging Britons, is perhaps emblematic of how unacceptable it was to even suggest it. The British, in particular, were fond of divining native law and legitimacy in order to cordon off certain groups of Africans as 'tribes' and thus separate them, and rule over them through some sort of identified leader. (Achebe evokes a fair bit of this in his Arrow of God, a worthy read if you have a chance.) This gave every person a place, in colonial eyes, and the few Europeans remained privileged. But British imperialists were also quite careful to avoid acting too blatantly in the non-settler colonies, so they ruled indirectly where they could via local leaders they guaranteed, and upon whom local complaints might devolve. They also were wary of handing over non-settler colonies to settler ones; looking back to Hyam's (now very old) Failure of South African Expansion and a few other treatments of the dream of a single South African dominion to the Zambesi, one point that comes up is that both 'modern' and 'traditional' leadership agreed on one point relative to entities like Bechuanaland/Botswana and Swaziland/eSwatini: handing them over to South Africa would lead to widespread rebellion, not to mention direct legal challenges that the Tswana (Bechuana) leadership, at least, had won in the past (1895/6). Basutoland (Lesotho) had in fact fought a war against a settler colony (the Gun War, 1880-81) to prevent full absorption by the Cape Colony and the alienation of land for white settlement; throughout, they made clear that they were still loyal to Britain. They got their way with the restoration of Protectorate from London in 1884, which is why Lesotho exists today. Heck, that's a big part of why all of those countries exist.

So yeah, settlers. One may add their economic domination via legal sanction and special connection to global markets, their tendency to interfere even more intimately with African social and cultural dynamics (or those of any non-European people wherever they were), and the almost bestial level of violence that accompanied resistance to them, which one sees in settler responses to First Chimurenga in Zimbabwe (Rhodesia), the Land and Freedom Army (Mau Mau) movement in Kenya, and so on. Of course, in those, the imperial authorities were quite happy collaborators.

So although I am not exactly sure what you mean by 'colonial structure,' the major distinction between those you name (and allowing for the complexity of 'Africa') is the presence of significant, permanently-transferred European populations, upon whom key rights solely accrued. India did not have this (although there were some settlers, and their story is a bit strange), nor did most of sub-Saharan Africa, although as time went on many in Asia and Africa would clamor for the same rights and representation in a fundamental challenge to the colonial monopoly on power, and in settler colonies, the unity of that power between empire and settler.

[edit/addendum: I suppose I'd toss Lorenzo Veracini's Settler Colonialism: A Theoretical Overview in here as a start, although the whole Settler Colonial Studies project has some controversy that swirls around it. But there's a lot of work on settler colonialism that gets at the insecure yet genocidal nature of the thing in a variety of places, including the USA.]