Culturally Europeans aren't vastly different from eachother, at least no more than any other similarly geographically situated group.

Well here's the misconception. You may not think they are culturally different (or "very much" so) but they certainly felt they were. The Germanic tribes in Roman times felt culturally separate and with a combination of difficult terrain they were able to throw the Romans out consistently. Napoleon's conquests lasted only a few years distinctly because those people were culturally separate from each other. They were a distinct people with a distinct identity who did not want to be conquered.

I mean if you look at cultural and political maps throughout the Medieval and Early Modern period you see Europe in varying stages of fractured and united but so is the Middle East and Eastern Europe. Just a cursory google search shows that the Indian sub-continent was also heavily fractured before European colonization. And you'll notice that as time went on, the scattered pagan tribes and burgeoning states in Europe began to coalesce while the West began to centralize even further. And then they would even further.

You are probably focusing on the Holy Roman Empire, who had hundreds of scattered states, but in a sense they were unified -- they all mutually defended each other and while every power around them was condensing into centralized, unified states they remained fractured. That was a major part of German unification, defense of the German people after the power void left after its fall, and as soon as that power void broke powers began expanding and centralizing those smaller powers.

So I'd contest the very premise that Europe was necessarily more inherently fragmented. It's just the rest of the world was incredibly sparsely populated. Even then though the fracturing of states had to do with major empires (Romans, Byzantine, Frankish in particular) breaking apart into competing states and over time they would gradually work toward centralization and larger, more unified states. However looking at population density maps, even modern ones with 7 billion people on this planet, you see just how sparsely populated the rest of the world, even most of Chinese and Indian territory, really is. Russia's population is almost wholly isolated to the West.

Really we should be asking what made China so unique that they managed to centralize into a single, enormous state while the rest of the world did not have that level of unity with so much population density. Because China is the exception, and very much not the rule. For that we should ask our Chinese experts, not us lowly European ones! :P