Did you mean a different dynasty? The Song dynasty reunified much of China in a much shorter time period, after the upheaval of the Five Dynasties and Ten Kingdoms period. The dynasty you might be thinking about is the Sui Dynasty, which reunified China and ended the tumultuous Northern and Southern dynasties era.

If you did mean the Song though:

Historically, Chinese historians and scholars have had a bias towards the civil side (wen) of history, and have more or less sidelined the military side (wu) of history. The Song dynasty is perhaps the epitome of this, with the cultural traditions and achievements of the Song being highly praised, while their military arm has been viewed by many as weak and underwhelming. It's easy to fall into this pattern of thinking, but the reality is that you don't just found one of the largest empires in history without having some military might.

The founding of the Song can be traced to Zhao Kuangyin (first emperor of the Song's) overthrow of the emperor of the Later Zhou, establishing his Song state. Now, overthrow was pretty common in this period. The reason we call it the "Five dynasties" is because, within the half-century, the rulers of North China had switched from Later Liang, to Later Tang, to Later Jin, to Later Han, and finally Later Zhou.

While conventionally, students might memorise the date 960 as the founding of the Song dynasty, it's important to recognise that the Song spent the decades after conquering and slowly reunifying China; at 960, the Song were rulers of the North. The Song didn't conquer the Northern Han until 979, nearly 2 decades after the founding of the Song. The Song also inherited a very advantageous position in the Later Zhou. Under the Emperor Shizong of the Later Zhou, the Later Zhou military expanded and ruled over a powerful and dominant state in the North.

Despite my earlier point about having some military might, the founding of the Song was as much a diplomatic/political war as it was a war on the battlefield. Unlike say, France and England warring in the same era, there was the overarching goal of reunification in the minds of many Chinese rulers. Many Chinese rulers in this period would style themselves as emperors and would name their states after former successful dynasties, and Zhao Kuangyin was hardly the first person to try his hand at reunifying China.

This is important: as the Song wished to rule over all of China, it benefitted them far more if they could simply convince the lesser states to follow them and reunify, rather than fight a bloody and costly war. And the other states in China at the time were not entirely opposed to the idea. The Song provided generous terms of surrender; after all, this wasn't a war between the civilised vs. barbarians! This was Chinese vs. Chinese, and many of these small rulers would later become part of the Song aristocracy once they surrendered. Wuyue surrendered and the rulers of the family were given lavish titles, with the last emperor ending up as Prince of Deng before his death. Later Shu for example, following capture of their military commanders, surrendered and the last emperor was named Duke of Qin. This approach only worked in the first place, however, because the Song had the military might to back up their demands and often resorted to shows of force and battlefield victories prior to capitulation.

So the Song won out because the progenitor of the dynasty overthrew the Later Zhou, which already held an enviable position militarily in the landscape of China at the time, and shrewd use of diplomacy and warfare helped the Song slowly conquer their remaining opponents.

Source:

The Reunification of China: Peace through War under the Song Dynasty. by Lorge, Peter. Cambridge University Press, 2015.