I think the saying is that China isn't the oldest civilization but rather the oldest continuous civilization. Even with China's modernization and radical changes in the communist era, their culture is still present to this day. Something that pretty much no other culture can claim (except for i think India and other countries in the Asia/Arab region) that aren't as influential and prosperous as China.

Just saying, the Vietnamese name for China pretty much means Middle Earth.

Firstly, no one is entirely sure when a "civilisation" in China emerged. Advanced political communities, metallurgy and a writing system were definitely present in China by the Shang dynasty (i.e. from around 1700-1500 B.C.), but the picture before that was far murkier. We have far fewer written historical records from that era than we do for middle eastern civilisations, because most writing was likely done on bamboo or other wood, as opposed to stone or clay as in Sumer. Archaeological records are also far scarcer - the geography of central China, with frequent earthquakes, floods and so on is not conducive to preserving anything of archaeological interest. It may be that in the future we find evidence of advanced societies in China that predate those of the Middle East, but we haven't yet.

As for the idea itself, I think its fair to say that it does contain a degree of Orientalist mystique, but also that it is perpetuated in large part by the Chinese themselves. The idea that China is the oldest continuous civilisation is central to modern Chinese nationalism and is part of a broader tendency to glorify the country's past (to the point of obscuring the diversity and lack of unity in early Chinese civilisation and the degree to which it has evolved since). The impact of nationalism on Chinese historiography is a huge topic that I probably shouldn't try to address here, but this book has a fair bit to say on the subject.