I think it's important to keep in mind that the colonies in South Africa, Zimbabwe and Kenya were very different from the colonies in India. Central Kenya and South Africa had temperate climates, and fertile soils for planting. These territories also had relatively low population densities at the time. So British settlers arrived, much like the did in North America and Australia. They (I am simplifying on a little bit here) declared all the best land to be owned by whites, and placed economic barriers to keep black Africans from competing. But these settlers planned on colonizing Zimbabwe long term. They, and especially their children, saw Zimbabwe Kenya South Africa as home.

The situation in India was very different from the start. First, the climate was hot and tropical. Nor was there much land that wasn't already densely farmed. I'm on my phone, so I can't look up the exact number, but the number of British administrators, soldiers, merchants etc was always shockingly low. It was Indians who largely staffed the British Raj. The few British administrators that did come almost always saw it as a temporary posting, and never made any plans to stay. Most of the children were sent to the UK for their educations. After independence, a small number of British administrators stayed on, but it was only a small number.