I answered this more or less in this post under a slightly different question: What stopped the British Empire from invading... Bhutan...?

More specific to "colonization" I would just add that it would be for the same reason that Britain didn't "really" colonize India. There were simply too many Indians and not enough Brits. It's not a perfect comparison to Bhutan, but in order for classic colonization to take place (and by that I mean to tame the native landscape, both physical, cultural, and religious) you need to be able to demographically outmatch the locals. As you can see, since Britain was kicked out of India, but the Anglos weren't kicked out of the Americas for the opposite reason, there were similar reasons in Bhutan.

Not to mention that you would need to convince settlers to journey there in the first place. Few white settlers were interested in settling down in the Himalayas which are an impressive and difficult landscape, even in modern times. Even without hostile - and at times organized - natives opposing you.

For a more detailed explanation on the history of British-Bhutanese relations, I point to my post above which is more or less a summary of Karma Phuntsho's "The History of Bhutan" (at least on that subject matter).