u/Aun-El has compiled a good list of answers related to this topic and while I’m seeing that a lot of them have some pertinence to the Chinese case, a lot of them focus predominantly on Europe. I’ll try to speak a little more to the China case here and actually I’ll be trying to problematize the idea of long-standing Chinese unity.

First, consider that Chinese political unity was overwhelmingly intellectual during the imperial period. While our periodization practices sometimes give the illusion of compartmentalized dynastic cycles, the periods between them were full of strife. Moreover, particularly in the later imperial period following the Mongol conquest of China, imperial families had to work very hard at making sure that rival claimants to the throne could not spring forth and take power. The Ming emperors, for example, enfeoffed secondary heirs and other royal relatives far away from the capital. The Manchu rulers of the Qing dynasty engaged in similar practices but in other ways kept the heirs in close competition and the ascension of one particular emperor, Yongzheng, resulted in the longtime house arrest of his brothers. Timothy Brook attributed this more combative approach to dynastic succession to a direct result of Mongol practices of tanistry—competition between heirs. (See: Brook, T. Troubled Empire) So if the empires of China were unified under the name of a dynasty, the turbulence that went on under that unity has to be accounted for as it colors our understanding of what the Chinese imperium actually looked like.

Next, the periods between dynastic successions, say, from Yuan to Ming for example, were long drawn out periods of disunity and civil war. The three kingdoms period, the slow collapse of the Song, and the Qing accession are two other famous examples with rival states cropping up all over the place and displaced emperors ruling as fugitives until being literally pushed into the sea (as was the case with the last Southern Song emperor). The violence that preceded and succeeded dynastic successions in China should cause us to view this concept of unity with a critical eye. Not least to talk about the rebellions within the borders of the empire that were unsuccessful, like the An Lushan rebellion and the Taiping Heavenly Kingdom.

That said, all of this violence was predicated because of the system of rulership that had largely been in place since the Zhou (1100-221) period. This style of rulership firmly focused authority in the legitimacy of one man in the form of the emperor and it did so by tying the emperor to the cosmos. Heaven, in an early Chinese conception, had a moral authority and the emperor acted as humanity’s interlocutor with Heaven by offering sacrifices. This privileged position (called the Mandate of Heaven) conferred the right to rule and emerged through the Shang and Zhou periods as these two royal societies controlled increasingly larger portions of the Central Chinese plain. (See Keightley: The Ancestral Landscape: Time Space and Community in Late Shang China (ca 1200-1045 BC)

Claiming control over the Mandate of Heaven (tianming) thus allowed for the passage of a new dynasty and of all the changes that happened between dynastic cycles in terms of governance—no two dynasties ruled in a congruent way—the Mandate of Heaven persisted from the Zhou to the Qing as the sole investiture of legitimacy. My perception is that while casaropapism began to form after Constantine, the West did not possess this same level of recognition for rulership in the same way. Additionally, I don’t want to confuse the fact that there may have been one emperor, but there were many many local and regional powerholders just waiting for their chance should it ever arrive, like young trees waiting for the dominant tree (the dynasty) to fall so they can have their turn at a crown. Those power holders not only propped up the legitimacy of an emperor, they also waited for their own chance to creat their own dynasty. It happened all the time. Moreover, just like in medieval and Renaissance Europe you have many rulers claiming divine right to rule, at times the Mandate of Heaven also had multiple claimants at the same time and I don’t want to confuse that either.

So, rather than thinking that China had a unified empire for 2000 years, which invites us to ignore the violence and turbulence inherent in rulership, it is usually better to conceive of each dynastic cycle as it’s own “fall of the Roman Empire” because that is essentially what it was: multiple Roman empires rising and falling in succession rather than one single unified empire that continued on end. There is more to say on the nature of Chinese dynasties employing the use of bureaucracy but I am typing on my phone and am getting tired. Suffice to say that the educated elites responsible for administering the dynasty’s basic functions did the lions share of the work in maintaining the perceived unity that this question asks about. Finally, to highlight how traumatic these dynastic shifts could be, there is a good book by Jonathan Spence called Return to Dragon Mountain that deals with the writings of a literatus named Zhang Dai as he lived through the Ming-Qing transition of 1644 and watched his friends and family make choices about whether to subscribe to foreign (Manchu) rulership or basically to commit suicide.

Tl;dr: China as an imperium always had cracks in its unity that were just waiting to be blown open.

While there is always more to be said, you might find the answers to similar previously asked questions interesting:

Why can't Rome divide and unify like China? by /u/ohea, posted two years ago.

A compilation of answers to similar questions composed by /u/qewryt, posted six years ago, including posts from the FAQ.

And a direct answer to the question /u/qewryt posted his list in response to: Why did Europe remain so divided after the fall of the Roman Empire, with no one being able to unite it like the Romans did? by /u/rsnowdon, also posted six years ago.