The term Hui or Huihui has been used more generally for Muslims and before that for Turkic peoples well before the modern dynasties. During the Yuan (Mongol) siege on the Southern Song, trebuches were brought from the Persia and were referred to in Chinese as 回回炮 huihuipao. The term Hui has had a very flexible history, sometimes meaning muslims in general, sometimes meaning foreigners, and sometimes meaning a specific group of Chinese who are now officially referred to as Hui in China.

The CCPs criteria are wholly impractical and not internally consistent. There are a great many groups (高山族 Gaoshan being one, 彝族 Yi being another) for which the linguistic and cultural criteria do not hold up. The 56 ethnic groups are a very poor attempt at demarcating cultures. Take it with a grain of salt.

That said, traditionally the Hui do have common history, being described as decedents of Arab traders during the Tang who married Chinese people and adopted the culture while retaining the religion. Mind you this is the tradition, not necessarily the actual history. However the traditional view in this case is consistent with the CCP's guidelines. Second, the geographical region at the time the Hui became officially recognised by the CCP was Ningxia, to this day still officially "Ningxia Hui Autonomous Region". A very large number of Hui have and still do live there. The 1950's recognition of the Hui as such was in part a political move on the part of the Communists. Now, despite Ningxia being pegged as the "traditional" homeland of the Hui, of course people in China migrate. The largest human migrations in the world are in China and happen every year. So there's your territory part.

To add to that, there's a long tradition of Hui identity as such among people who would today be described as Hui. This goes back to the Ming when Hui scholarship became somewhat widespread with a sort of Islamic renaissance with people like Wang Daiyu 王袋輿 and Liu Zhi 劉智.

And from my personal experience, Hui minority groups usually don't speak the same language.

Mandarin? Because that's the first language of the majority of Hui. Arabic was used for liturgical purposes and to some degree still is, and there is an Arabic script transcription system of Mandarin called xiao'erjin which was widely used by the Hui until more recently. But for the most part, they're still Sinitic speaking, and more typically, Mandarin speaking.

tl;dr: The Hui in fact very much satisfy the otherwise arbitrary definitions set forth by the CCP, much more than a large number of other groups in China. And even if they didn't, group identity isn't entirely reliant on government recognition anyway.