Thank you all for the responses. But my query was more India centric in the sense that what conditions led to so much fragmentation in the Indian subcontinent in terms of language, culture etc. (this is my first post on reddit so pls excuse me if I'm not doing it right)
Nooooo. A dagger to my heart!
Sinitic languages alone number around a dozen, arguably more. There's Mandarin, Wu, Cantonese, Min, Hakka, Pinghua, Hui, Waxiang, Jin, Xiang, Gan... But then within a few of those, Min for example, they can be broken down into 5 or more mutually unintelligible dialect groups which are languages in their own right.
Add to that Tibetan, Uyghur, Zhuang (all on the money, by the way) as well as smaller languages like Hmong or Thai, and China's actually quite a diverse landscape, linguistically. Then on top of that you have multiple Korean dialects, Russian, Xibe (Manchu, basically), and the list goes on. That Mandarin is the lingua franca doesn't detract to the underlying diversity that's very visible in even just a short amount of time here.
And just to add to that, populations of non-Han ethnicities (and therefore their language use) are under-reported.
Anyway, this is largely an issue of perspective and an uneven exposure to the two countries' linguistic makeup. If you can add some detail to your question I'll happily spend a good chunk of time clarifying my own answer. As it is, your question is somewhat vague, and the initial question is based on a misconception, albeit a wide-spread one, so if you'd like a deeper more detailed answer the question will need to be added to somewhat.
edit: Left a sentence half-finished. Oops.
India and China both possess a significant number of languages.
If you take a look at Ethnologue for India and China, you'll see that India has ~460, and China ~300. So, call it 160 languages more in India than China. Out of those, India has more extinct languages than China does - so China's doing better on the retention level for their variety of tongues.
In linguistics, there's a phenomenon observed with numbers of language and geography: namely, that geographical features such as mountains and rivers will halt the spread of languages and dialects; it's why places like [Papua New Guinea will have 800+ languages](https://www.ethnologue.com/country/PG]! Take a look at this map of India and this map of China ; you'll find that the majority of these languages that are "small" and help add to India's numbers are in the mountainous areas - and that China's major varities of languages happen around their mountainous regions, too. Geography is a large contributor.
I'm sure /r/linguistics would be able to provide a better answer (I'm drawing on my years of linguistics in uni for this, while I try to track down any actual books that would address this specifically).
I live in India and have friends in China, and I'm also inclined to hypothesize that some of it has to do with cultural retention vs homogenity: India has a large number of official languages (22), and China has only one. India's individual states virtually all have their own language, and it is not uncommon for a child to grow up with a mother tongue, and speak a state language and Hindi and English as well.
Actually, as a person learning from Chinese from a teacher who makes a point to teach us about China and has been to China, the nation has many, many languages and dialects.
Shanghai has its own dialect, so does Beijing, the Southern region its own language (Cantonese). Then there are the many native languages in YunDong and GuangDong Provinces, not to mention the different languages in Xinjiang and Tibet. You just don't hear about them so much because everyone learns Beijing Dialect because that's what all the TV broadcasts are in.
Beyond the semantics of your question, which others have covered in detail, the question seems to deal in part with the perception that, in most parts of the world, you can say "Chinese language" and people will assume you're talking about Mandarin, or maybe Cantonese. However, saying you want to speak "Indian" will get you funny looks.
Most of what I gather is the difference is based off my reading of Fukuyama's Origins of Political Order, which runs the risk of being dubious pop history, but I'll go with it. Basically, the idea is that China has historically tended towards being a unified state. Certainly there were things like the Warring States period, and many parts of what is now China entered and exited its sphere of influence over time, but the concept of 'China' as an unified political entity that should exist has been around for a long time. While there are many dialects and local languages, the longevity of the concept of China meant that Mandarin came to be dominant.
In contrast, India wasn't regularly unified in the same way. There were a few periods of unity, even then the whole subcontinent never had one ruler for very long until the British came in, but mostly the trend was a large number of independent states. As such, no one language came to dominate politically like in China.
Another reason is the evolution of the Chinese written language, which unlike most languages in the world today, does not have written alphabets that somewhat correspond to phonetics.
In Chinese history, as the main Han Chinese ethnic group encountered other ethnic groups and other languages, they would often adopt new words and create new Chinese characters that are phonetically similar to the other language, in order to make conversation easier.
(The Chinese do this even today, creating new Chinese words for English words that cannot be translated easily).
As many of the minority ethnic groups in China (in ancient time) did not have their own writing systems, the Chinese writing system (and the spoken words) became more in common use.
This creates an appearance that the Chinese language (especially the written system) is 1 single system, when in reality, the Modern Chinese language evolved from a merging of many many (100's) of local languages.