PART 1

There's a lot of debate about the exact nature of the Anglo-Saxon takeover of what is now England, so I'll present some of that and then give my take on it in the hope that it helps to answer your question.

The classic version of events that most English schoolchildren have learned is the expulsion theory, namely that war bands of Angles, Saxons, and Jutes took over the southern part of Britain, wiping out the natives wholesale and inviting their relatives and friends across the sea to settle in their place. This theory is derived primarily from the writings of people such as Gildas and Bede. However, it has fallen out of fashion nowadays, largely due to the implausibility of such a successful, organized attack, and the fact that archaeological evidence does not suggest such a complete massacre.

Another theory that seems to be gaining some ground is the warrior elite model. In this version of events, a small number of Anglo-Saxon warriors settle in Britain rather than a large-scale migration. The majority of the native Romano-Britons survive, but are forced to assimilate to the culture of the politically dominant but numerically insignificant Anglo-Saxons. This theory is, in my opinion, problematic, for reasons I will get to in a moment.

DNA tests have been done to shed further light on the issue, but they have varied wildly in their results. One study postulated that something like ninety percent of the DNA of the English comes from the invaders from northern Germany; another claimed that the Anglo-Saxons had less of a demographic impact than the Danish Vikings. One potential complicating factor is that a lot of the DNA in Northern Europe is quite similar, regardless of language. Genetic science is advancing, and might be able to give a clearer picture in the future, but as of right now it still leaves us in the dark.

But to get back to the main topic, you actually lay out a massive problem with the elite migration theory in your question. Such groups may influence the cultures of the areas they rule over, but rarely manage to affect a complete cultural transformation in the way an elite band of Anglo-Saxons would have to have done. Much more often they end up being assimilated themselves. One notable exception to this is the Romans managing to spread Latin across their empire. However, the Roman Empire was very unique at the time--it was an extremely powerful, centralized, well-ordered society with massive infrastructure. The Germanic tribes that lived on its periphery were none of those things. And, indeed, we see that upon conquering formerly Roman territories, groups such as the Lombards and the Visigoths, to name just a few, all wound up speaking Vulgar Latin and being assimilated to the populations they ruled over. Even the Franks, who were dominant enough to give their name to the region they ruled over, ended up taking on its Gallo-Romance culture and language.

Languages can tell stories that written history leaves behind. Even if something like the spread of Latin does occur, there tends to be a substrate influence from the conquered people left upon the new language spoken there. For example, it is widely agreed upon that substrate features of Gaulish exist in French, including several common words as well as grammatical features. Now, there is a school of linguists that subscribe to "the Celtic hypothesis," which is to say that there is a Brythonic Celtic substrate in the English language, but their ideas have not gained wide acceptance. It doesn't help that the majority of the things they claim to be features of such a substrate don't really start showing up until a thousand years after the Anglo-Saxons began migrating, a time gap which is, frankly, rather ludicrous. Much more logical explanations (in my opinion, at least) exist for most of these features that don't involve Celtic languages at all. There's also the fact that English has remarkably few loan words from Brythonic, and those that do exist are for the most part fairly obscure. And the fact that most "domestic words" in English do originate with the Anglo-Saxons suggests that whole families migrated, not just a few men.