It varies case to case and is too complex to answer properly in a short statement. I will try and provide a few examples to demonstrate this.

The British military hold of their possessions was often referred to as "the thin red line" for a reason. On a map, the Empire was vast and impressive (colored red). Reality on the ground was much different. They relied heavily upon naval hegemony over actual ground troops. In India for example, they had many local troops known as Sepoys that outnumbered British regulars. Additionally, Plassey marked the end of East India Company control and the transition to India as an imperial possession. They than integrated Indians within the lower bureaucracy and provided education. While they were by no means benevolent, the empire was able to prevent revolt partially through being "not as terrible as the EIC." Basically, you could be a total dick as long as you didn't push your subjects to the point of revolt. There are also other factors. India was not united by any means and buying a Prince or two could go a long way to achieving control.

The Spartans controlled an immense population of helots (basically serfs) with a very small population. They always feared an uprising and often did not campaign far from home due to this. They were very strict and often brutal to their subjects to keep them in line.

If you look at the European Imperial era (19th century), it becomes clear that there are many cases where few men hold vast territory through sheer brutality. The smaller the power, the more brutal they generally were. Look at Belgian Africa. They had far less resources and manpower than other great powers and thus relied on brutal force.