My understanding is that they actually didn't have a very large occupation force, and were rather unprepared for the rebellion, and that the rebels capitalized on this by overwhelming the increasingly isolated French outposts over time, with the outcome of the Battle of Dien Bien Phu being the end result. Despite the lack of a real investment of forces that seems to indicate they managed to hold onto it from the mid-19th century until after World War II except for the Japanese occupation, though. Why is that?
France conquered Vietnam in 1885 mostly by virtue of superior military technology and a fairly ineffective Vietnamese monarchy. The emperor at the time (Tu Duc) did not understand the danger that the French posed, and the Vietnamese military was horribly outmatched. France had steamships, machine guns and modern artillery. Vietnam had spears and cannons that fired stone balls. Further, Tu Duc eventually gave up fighting, hoping that cooperating with the French might preserve some measure of independence for Vietnam. That did not happen.
An anti colonial uprising broke out almost immediately after France seized Vietnam. It was called Can Vuong, or "Restore the King," and was mostly located in Northern Vietnam. As the name indicates, Can Vuong tried to kick out the French and restore the monarchy to prominence. The French were able to crush this movement by 1889. Thereafter they ruled with an iron fist.
It's true that France never had many troops in Vietnam. They maintained control in a few ways. First, they split Vietnam into three parts - the north was called Tonkin, the center Annam, and the south Cochinchina. They scrubbed all mention of Vietnam as an independent nation from the educational system. Vietnamese people were no longer to consider themselves Vietnamese. Rather, they were Tonkinese, Annamese or Cochinchinese. France portrayed itself as essential to Vietnam's development - France was the mother country. They also did their best to coopt the Vietnamese upper class, who were mostly intellectuals and regional politicians known as mandarins. Despite this effort, some Vietnamese intellectuals spoke out against French colonialism. Probably the most famous was Phan Boi Chau, who wrote a book called Viet Nam Vong Quoc Su or History of the Loss of Vietnam, in the early 1900s. This book laid out that Vietnam had once been independent and also that France had conquered it. Needless to say, the French considered this book very dangerous, and Phan Boi Chau spent much of his life either on the run or under house arrest.
France also maintained control through extremely high taxes and the corvee system. Peasants had to borrow money from French officials to pay their taxes, which meant that they were caught in a never ending cycle of debt. Under the corvee system, most peasants were required to spend a certain amount of time (say a few months) as unpaid laborers. And France also used native levies and informers, much like the British in India. Until the Viet Minh, all uprisings were easily localized and isolated. What made the Viet Minh special was their ability to build a national network across all of Vietnam, making it very difficult for the French to stamp them out.