Why is this claim always made with regards to China?* I have a very limited understanding of Chinese history, but I find it hard to believe this kind of claim. The website seems to attribute this claim to China's written history, but hasn't the writing system changed over 3,500 years?
*Link here because it contains a ) and I can't format it properly: https://www.historians.org/about-aha-and-membership/aha-history-and-archives/gi-roundtable-series/pamphlets/em-42-our-chinese-ally-(1944)/the-oldest-living-civilization
As the source told you, China is generally considered the "longest continuous civilisation" in the world today. The earliest uncontroversially attested centralised Chinese polity was the Shang Dynasty in 1500BC. By this point, Egypt was in its 19th dynasty and the peak of its power, so it's not unfair to say that China was certainly not the first 'great' civilisation. It certainly predates the Romans though, and arguably the Classical Hellenistic civilisation as well.
The difference is that the Chinese civilisation has basically survived unscathed since then, while all the above have been displaced over time. Shang Dynasty China was obviously extremely different from modern China, but the writing system developed during the Shang Dynasty is essentially completely legible today. With minimal training, a well-educated native Chinese speaker could easily learn to discern between the characters used during the period. What has changed tremendously is the phonology, but because the Chinese writing system does not capture phonology, two people might be able to correspond perfectly in writing, but have no clue what the other was saying. The effect of this is that there has been no real need to modify the writing system, except in terms of calligraphy, to make it easier and more codified to write. That being said, over time people came up with their own characters for different words so it has been a project of many emperors to publish definitive dictionaries every now and then which standardise the way the words are written. The Communist government did this most recently in the 50s, resulting in what we call Simplified Chinese today.
The early Shang Dynasty was probably not as advanced as Egypt, but we do have archaeological records of everything from astronomical observations, military guidebooks, and political philosophical treatises. We also have evidence of grand public works like massive roads, fortifications and canals, which would have rivalled the type we saw in Egypt at the time. We know the Shang Dynasty population was approximately 30% urban, which implies an economy obviously centered on agriculture, but with significant manufacturing and services components. Based on a variety of archaelogical estimates, historians generally put the Shang population at anywhere between 3-10 million people, which is at least the same as Egypt during Ramesses II. Chinese civilisation in the precise form you know it today was probably defined around 1000BC during the Zhou Dynasty, when you get all the precise philosophical forms like Taoism and the beginnings of what would become Confucianism, as well as a fully codified class of scholar-bureaucrats. But all these had their roots in the Shang Dynasty.
Are there extant civilisations which can claim the duration that China claims to have? The best contenders are the Copts in Egypt, since the Coptic language is essentially a descendant of Late Egyptian. However, apart from that, the Copts are nothing like the Ancient Egyptians. For one thing, Coptic is written in Greek, and secondly, Copts are defined ethno-religiously by their particular brand of Miaphysite Christianity.