There is no real rule of thumb. It depends on the balance of power and the prestige of the various languages, although population size can have an influence. For instance, Gaul was romanised because the Roman culture became the more prestigious one in the region. So first the local elite started speaking Latin, and then it seeped down to the rest of the population. By the time the Franks came, Latin was the still the language of prestige (since the Roman empire still existed), and Latin was also the language of the majority of the population, so they adopted Latin.

In Britain on the other hand, Latin had never rooted as much, and the Anglo-Saxons brought a new culture of prestige, so over time their language became the prestigious one.

Ultimately, it is impossible to predict, as it always is with language change. Prestige and population size are important, but it also depends on how stubborn people are, how much communication there is between the speakers of the different languages, how much a ruler might press the use of his native language, and all kinds of factors.

This has to do with the needs of the empire and its ability to enforce social policy on that level. It looks like a coin toss because different powers envision their power differently, and are differently able to raise their preferred cultural forms to prestige status. The Romans were more about centralization of a certain kind of culture, and to this day certain former more stable parts of that empire continue to speak Romance languages. The Ottoman and Qing Empires saw their rulers as universal over humanity and felt no need to justify their power culturally, so there was no requirement for cultural integration among the subject peoples. In the Ottoman case, Arabic was prestige but neither Arabic nor Turkish was forced on, say, the Albanians. The Qing had five official languages, one of which- Manchu, the "language of the conquerors"- effectively died out before the empire did. It just wasn't as useful to their authority as, for example, English was to United States rule over Native American populations or Spanish to Spain's possessions in South America. The PRC, which was the eventual successor empire to the Qing, is at present rather successfully pushing Standard Mandarin on its subjects because conformity of culture is infinitely more crucial to nationalist, Party-centered power design than it was for the Aisin Gioro. The Japanese Empire attempted a fascist-type annihilation of subject cultures but couldn't hold on long enough to regions like Manchuria, Taiwan or the Korean Peninsula, or with sufficient legitimacy, to force the people in these colonies to adopt the language of their conquerors. Some empires just don't feel the need to exert power in this sphere, others do.

It varies from conqueror to conqueror. It's a trait of the Normans that they tended to integrate and merge with the 'conquered' rather than stamping Norman domination upon them. The Normans adopted various Anglo-Saxon administrative practices. The writ, for example, is most likely an Anglo-Saxon invention that was then adopted and propagated by the Normans (I can cite articles on this when I get home, if anyone wants sources. Mark Hagger has written some but the titles elude me. EDIT: The Earliest Norman Writs Revisited). Language was another symptom of this approach.

Similarly, in Sicily the Normans adopted local practices such as using a lay notarial class (instead of a clerical one) and certain tax practices. It's not entirely clear what the most common spoken vernacular was in Norman Sicily but there are indications that it was Arabic, and secondarily Greek.

My point is that different conquering powers do it in different ways, so there's no general rule. The Normans integrated, others didn't. You could argue that it's a question of pragmatism. The Normans were very pragmatic; they took the most useful parts of the conquered peoples, including the vernacular language(s), and made them their own.