This is actually a question more about how one defines ethnicity rather than about languages.

Han as a word has been around for a long time, but the way it has been used has changed. Han in its modern usage comes mostly from the nationalist period of China. In the process of trying to create a modern nation-state, the people that led the movement (Sun Yatsen, for example; writers such as Liang Qichao) based the nationalism in racial terms. The last dynasty of China’s imperial period was the Qing Dynasty. The Qing were Manchu rulers and had a much different cultural history than the people further south and in fact spoke completely unrelated languages to the Chinese languages. The Qing did, however, uphold the scholar-official Confucian bureaucratic government and for the most part allowed the people they conquered to continue with their cultural traditions (think Tibet, for example).

The latter part of the Qing Dynasty was a tough period for many reasons (Taiping Rebellion, Opium Wars, the Scramble For Concessions, etc.). Basically the Qing were hemorrhaging land and resources and “sovereignty” to the West at the end of its reign. The nationalist movement was powered as a reaction. For “true” Chinese to form their nation (a 中華 “zhonghua” nationalism). 漢 (Han) was used in this context of nationalism. Who was Han though? During this time Han became a political tool. A designation for various related, but distinct, ethnolinguistic groups in China that shared some kind of culture. Contrary to popular belief, China was not and is not homogeneous. It’s actually very diverse. The regions and language-groups had different lifestyles, foods, traditions, etc. There were some prominent aspects such as writing that had become shared over time for various reasons.

This movement was meant to take away China from the perceived Manchu invaders, but it really had to create a sort of new identity. Zou Rong, for example, spoke of China being for Chinese people and that “our compatriots must all recognize themselves as the Han race, the Chinese people, and China”. This is not speaking as if the Han had lost China and were getting it back; rather Zou is telling the people that they needed to come together as a Han race. A lineage was created during this time to “show” how all these related ethnolinguistic groups were actually coming from some kind of common ethnicity. There is a drawing in James Leibold’s “Searching for Han” that shows this lineage. The argument is essentially that “Han” in its modern sense is an imagined community (look to Benedict Anderson for further reading on what this is specifically) not totally unlike an “American” or “Indian” identity or any other nationalities, for example. A feeling of togetherness despite not literally being near each other. So think of your question again using the “Indian” example: “why is it that Indians have so many languages despite being one ethnic group?”. The answer is that “Indian” is a type of nationality or imagined community more than a single ethnolinguistic group. The same is true of Han. Ethnicity or race in general is something ascribed to someone either through personal choice or by others. Han in its modern conception is mostly a political tool. Leibold even notes that several scholars point to Han as being a tool of oppression in the modern mainland China nation-state.

In addition to the post regarding the nature of what is considered Han, I would like to add that for many of China s historical conflicts, both internal and external, is marked my exodus out of the Chinese yellow river heartland. This means that Sinitic languages are exported to various regions in waves.

In the south east were rivers and mountains create semi isolation for speakers , the dialect begins diverging from the source material. When peace and stability is reestablished , continuous wave of literary culture from the cultural heartland ( insert any unified dynasty) so the Sinitic language of the literary class that has evolved separately from the regional variants becomes stapled on as an addition to the regional variety.

An example if this would be the hokkien dialect. The basis of the dialect starts from old Chinese from the han and Jin periods plus local indigenous sub stratum .

Colloquial speech today in Fujian, Taiwan , Singapore, Philippines, and maritime SEA reflect this so the character 人 is pronouced as "Lang" rather than mandarins "Ren".

But.. after China was reunified under the early medieval period of the Tang. New migrants and assimilated people begain speaking middled Chinese rather than old Chinese based language s. This became hakka,. Gan, and Cantonese. The region of fujian as well imported tang pronouciations are part of the educated literary readings in addition to their colloquial pronouncations so 人 is pronounced "lang". As well as the middle tang Chinese sounding "Jin". This is similar to how English has. The Anglo Saxon Germanic work for " folks" " volks" as well as the Latin based Norman word "people's"

Imagine this process applied to every regional accent and dialect. This will give you an idea for why the variation exists.

The Sinitic language s are less like dialects and more like a language family similar to latin and romance languages. Loosely related but share a common ancestor.