I can answer for Vietnam in particular. The penetration of the French language was lower because the Vietnamese literacy and urbanization rates during the colonial period were low. Primarily city-dwelling cultural elites spoke French because they were brought up through the French school system.

During the American build up in the 60s and 70s, English became a much more practical language to learn (at least in the South), which continues to today where English is taught in schools because of its international importance in tourism and trade.