I'm also thoroughly persuaded by the arguments that it's time to retire the term "Anglo-Saxon," as a disclaimer.

But, the term appears in a small handful of early medieval manuscripts. It was a term used in the period, including by Paul the Deacon (an Italian, known for his work Historia Langobardorum) but it was not the primary term used, nor does it seem to have summed up how elites in the period conceived of themselves.

The term begins emerging as a scholarly term in the 16th or 17th century. In a donation to Cambridge University in 1571, Parker includes a manuscript of 200 leaves, which includes a "historia Angliae Sax." The manuscript is now damaged, including the relevant section, but is now Hh. 1. 10. Abraham Wheelocke is another early figure, publishing an early (and very incomplete) edition of the so-called "Anglo-Saxon Chronicle" in 1644 under the title "Chronologia Anglo-Saxonica." Wheelocke was an immensely talented linguist (he was a professor of Old English, Latin, and Arabic) and a deacon of the Church of England. I can't find an edition to see why he uses that term, sadly.

"Anglo-saxon" as a broad term for the period does not appear to become generally accepted until the 19th century. Even in 1823, Rev. James Ingram refers to the "Anglo-Saxon Chronicle" as just the "Saxon Chronicle." However, by 1837, the term "Anglo-Saxonist" emerges to refer to a scholar of early medieval England. This appears to generate some idea of a "pure" culture, and the OED records in 1879 the use of the term to refer to someone who is holding to some idea of English cultural purity (Ellard 2019, 21). The bleed from one to the other was accelerated the relative success of prominent 19th century scholars like James Douglas and Sharon Turner and their use in justifying and upholding imperialism. Even though the academic use of the term predates the broad cultural use of the term, the sensation that this refers to a group that is "uniquely" white and English is intimately tied to the study of this medieval past that is, in some ways, mythic.

As much as older scholars will sometimes argue otherwise, the term has never shed its racist associations; the medieval culture is still used as part of a myth of cultural isolation and whiteness. Dr Mary Rambaran-Olm (@ISASaxonists on Twitter) has documented lots of cases in her public-facing work of accusations of "not belonging" she has received for being a Black scholar of early medieval England. Her experience is in no way, shape, or form unique, and is revealing of how the term has shaped the perception of who "owns" the past and who gets to study it. As such, I think it is safe to say that, regardless of whether Wheelocke meant it as a racial term, it became inherently tied to whiteness in the 1800s and has been ever since.

As to whether a department like ASNAC should exist or not.... that's a much more nuanced question. Donna Ellard argues that the term "Anglo-Saxon" isolates scholars of early medieval England from entering into the discourse of the "global Middle Ages," though lumping England, Brittany, Ireland, Scotland, and Scandinavia into one umbrella, as ASNAC does, alleviates this problem somewhat. Certainly, the period is worth studying and there are so many interesting things being researched, it's just a question of how to make that as inclusive as possible and reflect the inter-departmental and inter-disciplinary methods that are vital to modern academia. A benefit of a separate department existing is that it ensures that funding goes there separately, enabling research to continue without having to directly compete against scholars at Cambridge doing interesting work on other time periods and other parts of the world.

P.S. I am struck here again how Thomas Jefferson proposed that the legendary leaders of the "Saxon invasion", Hengist and Horsa, should be on the seal of the United States, on the grounds that they were the "origin point" of the cultural norms that led towards parliamentary democracy, and therefore the laws that the US adapted. The Saxons as a marker of cultural uniqueness and superiority predates the emergence of modern scholarship on the subject, even if the term "Anglo-Saxon" didn't explicitly entangle both senses. But that is slightly tangential to the question you asked, so I'll leave it down here.

Further reading: Donna Beth Ellard, Anglo-Saxon(ist) Pasts, postSaxon Futures. Punctum Books, 2019.
This is a fairly detailed analysis of 19th century British scholars' roles in defining "Anglo-Saxon studies" and the ways that entangled race and imperialism, and includes a solid overview of the modern discussion around the appropriateness of the term. The book is available chapter-by-chapter on JSTOR if you have access.

This is a really good question, and there's divide in the field as to whether the term should be used or not. Personally, as someone who studies the period, I don't have an issue with not using the term; I just say early English history or early medieval England when explaining my specialty. It is ahistorical, I think there's only one example from the period where the term appears and it definitely was not a common term used by the English to refer to themselves, so clinging to it as a historically accurate term is a weak argument. There's also been a lot of feedback from scholars of color who have been made to feel unwelcome or like they don't belong because they don't 'look' like someone who 'should' be studying 'Anglo-Saxon' history, which is obviously racist and a huge problem. I still need to explain the precise time period I study most of the time anyway, regardless of the terminology I use, so there isn't even the benefit of clarity. To my mind, there are strong reasons to get rid of the term and not enough justification for keeping it.