First, I would like to break this down into four periods and examine each individually, as there was a series of changes. The first period is the Qin-Han Period (221 BCE-220 CE), the second period is Cao Wei to Five Dynasties (220-960 CE), the third is Song-Yuan (960-1368 CE), and the fourth is Ming-Qing (1368-1911 CE).

Qin-Han:

During the Qin-Han period, there were three officials who ranked above the rest. Collectively, there were called the 'Three Ducal Ministers' or 'Three Lords' (三公, san gong). The Three Ducal Ministers first appeared in the Zhou Dynasty, but by the Qin, the positions had changed.

During the Qin, they were the Prime Minister/Chancellor (丞相 cheng xiang), who assisted the emperor with all political matters; the Grand Marshal (太尉, taiwei), who commanded the empire's military, and the Imperial Secretary (御史大夫, yushi daifu), who handled all memorials going to the emperor and sent out imperial edicts.

The Western Han followed the Qin model, but during the Eastern Han, the three were changed to the Minister of War (大司馬, da sima), Minister of the Masses (大司徒, da situ), and the Minister of Works (大司空, da sikong). Together, these three officials directed the 'Nine Ministers' (九卿), who were in charge of more specific things.

Cao Wei to Five Dynasties:

This can further be subdivided into Cao Wei to Sui and from Sui to Tang. After Cao Cao seized control of the last Han emperor, he was appointed Prime Minister and appropriated power to himself. Traditionally, the court had two Prime Ministers. The Left Prime Minister was the more powerful position while the Right Prime Minister served as his assistant. Usually when someone is referred to as Prime Minister in historical texts, it meant they were the Left Prime Minister and directed state affairs. The Three Ducal Ministers survived as the de facto highest positions in court, but they increasingly became ceremonial and prestige positions that carried very little power, though sometimes they were held concurrently by the Prime Minister. The position of the Prime Minister survived well into the early Ming, though as we shall see, it underwent some transformation.

The Sui Dynasty is a curious case, as the 'Three Department and Six Ministries' (三省六部) actually matured in the Sui rather than in the Tang, as popular history tends to suggest. This officially replaced the 'Three Lords and Nine Ministers' used by the Han (the Three Departments were actually formed as early as during the Qin and was used and developed by subsequent dynasties, but it was only during the Sui that they were officially institutionalized and copied by later dynasties). They were the Department of State Affairs (尚書省), the Central Secretariat (中書省, in the Sui it was the 內史省), and the Chancellery (門下省). Of these three, the Department of State Affairs directed the Six Boards (Personnel, Finance, War, Punishments, Rites, Works) and they were headed by the two Prime Ministers, now known as Left and Right Puye (僕射).

The Tang largely copied the Sui system, but the head of the Department of State Affairs became the Minister of State Affairs (尚書令). However, because this title was given to Li Shimin, who later became Emperor Taizong, it was never again given out and so the de facto head of the Department of State Affairs was once again the two Prime Ministers (puye). To keep the power of the Prime Minister in check, the Tang emperors appointed multiple "Prime Ministers" by raising the ranks of the heads of the other two Departments and dividing up the puye's power among them.

The Tang system was used the various dynasties and kingdoms of the Five Dynasties and Ten Kingdoms period (907-960 CE).

Song to Yuan:

Coming out of the Five Dynasties and Ten Kingdoms, the Song emperors were very much afraid of power concentrating into the hands of one official, as they believed that was the reason why the Tang fell. With this in mind, they proceeded to split up the power of the Prime Minister amongst many officials. Prior to the Yuanfeng Reform of 1080, the Three Departments existed in name only but no one was actually appointed to those positions. The most powerful official (in theory) was the Prime Minister (宰相, zaixiang), but the Song emperor appointed a Vice Prime Minister (執政) to keep him in check. Furthermore, control of the military was given to the Privy Council (樞密院) while finances was controlled by the 'Minister of the Three Departments' (三司史, sansi shi), who headed the three most important financial organs of the empire. In this way, you can see that the Prime Minister actually had very little power.

In 1080, Emperor Shenzong reformed the administrative system to follow the Tang model, reinstating the Three Departments and the Six Ministries, including the two Prime Ministers. In 1130, after the court moved south, the pre-1080 system was reinstated. This was changed again in 1173, when the 1080 reformed system was reinstated. The only difference was the Left and Right Puye were changed to Left and Right Prime Minister.

The Yuan got rid of the Department of State Affairs (尚書省) and the Chancellery (門下省), keeping only the Central Secretariat (中書省) as the highest state organ, controlling the Six Ministries. The Left and Right Chancellors headed the Central Secretariat, with the Left Chancellor being the more powerful of the two.

Ming to Qing:

The early Ming maintained the Yuan system, instituting the Central Secretariat and the Two Prime Ministers. In 1380, Zhu Yuanzhang killed the Prime Minister Hu Weiyong and purged the government, and he officially got rid of the Central Secretariat to take direct control of Six Ministries. To ease his workload, he set up a coordinating agency known as the Grand Secretariat (內閣), composed of Six Grand Secretaries (內閣大學士) drawn from the Hanlin Academy. At first, these Grand Secretaries had very low ranks (5a if I remember correctly), so as to not make them overly powerful. But the problem was these Grand Secretaries handled all memorials and had close access to the emperor, so even though their rank was low, their power was considerable. During the Yongle era, (1402-1424), this system was further refined and the term 'Grand Secretariat' officially came to be used during his reign. The Grand Secretariat gained considerable power during the Xuande reign (1425-1435), when they were given the power to post rescripts to memorials (票擬, piao ni). They screened and scrutinized all memorials going to the emperor and decided on a proper response, posting it on a rescript to the front of the memorial, before forwarding it to the emperor. In time, the Grand Secretaries began concurrently holding other ministry positions and the head of the Grand Secretariat (內閣首輔) became a de facto Prime Minister.

But the Ming is also interesting because it was dominated by eunuchs. Both the Jiajing and the Wanli emperors did not attend court and rarely met with officials, instead relying on eunuchs as intermediaries. The Si li jian (司禮監) had the power to approve or reject memorials (known as 批紅), so the head eunuch (司禮監掌印太監) was also incredibly powerful, functioning as another de facto Prime Minister.

Like the Ming, the Qing did not appoint any Prime Ministers, nor did they use the Grand Secretariat. The early Qing had the Grand Council of Princes and High Officials (議政王大臣會議), composed of eight Manchu princes and a few Manchu officials, which was later expanded to include Han officials. The Kangxi emperor relied on a group of officials who met in the Southern Study of the palace, so the Southern Study (南書房) replaced the Grand Council of Princes and High Officials and the Grand Secretariat in importance. These officials were known as "[having] access to the Southern Study" (南書房行走). Later, the Yongzheng emperor formed a small group of trusted officials to discuss military policy, known as the "Office of Military Secrets" (軍機處). Though at first it was only a place to formulate military policy, it later began to take on administrative duties as well, much like a Privy Council, which is why it's known as the Grand Council. In this way, the Grand Council eclipsed the Grand Secretariat as the highest decision making organ in the empire. The head official was the Chief Councillor (領班軍機大臣), who like the Chief Grand Secretary, acted as a de facto Prime Minister.