I recently read Pu Songling's Strange Tales From a Chinese Studio and this exam system seems wildly intricate and profoundly important to the elite of the time. I'm wondering how it came about, how it came to occupy such an important part of high society, and if there are any traces of it in contemporary Chinese society.
There is a common misconception that the imperial civil service exam blossomed with the Tang. This is false. Prior to the Song, the government was usually dominated by aristocratic families with illustrious lineages going back centuries. Naito Torajiro, who pioneered the theory of the Tang-Song transition, basically stated that before the Song, the emperor and his family was more of a "first amongst equals" and shared power with his aristocrats, some of whom actually had the power to dethrone the emperor.
Because of this, the prior method of recruit prior to the Song was through the yin privilege, where fathers who were ranking officials could recommend his sons or other close family members to lower positions. The other method of recruitment was, of course, by utilizing connections with officials who could recommend people to government. The concept of an examination to choose officials actually started during the Han. When Han Wudi established Confucianism as the leading state ideology, candidates were tested on their knowledge of the Confucian classics and the emperor selected talented people based on their responses. Still, the number of officials this method produced was minuscule, as recommendations and connections continued to account for the majority of officials entering the government. Furthermore, the exam was mostly used to classify candidates who were already recommended. From the Three Kingdoms up until the Sui, new officials were mostly recommended by ranking officials or by local elites.
The Sui was the first to establish a specific category of official chosen by exclusively by an examination system and the Tang inherited this system. Empress Wu Zetian in particular heavily developed this system, as she had usurped the Tang throne and therefore needed to build up a base of support that she can rely on. The civil service exam was thus greatly expanded and the more degrees were also handed out. But this system was still very restrictive - candidates needed to travel to the imperial capital to take the exam and the review process was not blind. This allowed room for manipulation. Furthermore, the bureaucrats produced as a result of this system did not displace the existing aristocrats.
The fall of the Tang and the subsequent chaos of the Five Dynasties and Ten Kingdoms put an end to the great northern aristocratic houses. The Song dynasty thus began to rely on the civil service exam to draw potential officials. The civil service exam we know of today was really developed by the Song. For example, the review process became blind and scribes copied the responses before submitting them for review to prevent the reviewer from recognizing the candidate's handwriting. Lower level exams were also held in the prefectures and counties. The exam was held once every three years. These practices were all inherited by later dynasties. The Song issued out a lot of jinshi degrees, much more than the Tang and I believe more than the subsequent Ming and Qing. Still, studying for the examinations was expensive and difficult, and only wealthy families could afford it. The result was that it created a new form of aristocracy, one dominated by a wealthy gentry class rather than lineage.
The Mongol Yuan initially did not hold the civil service exam. It was instituted in 1315 for a couple of years and then discontinued, before being instituted again in 1342. But the Mongol exams were skewed heavily in favor of the Mongols. Mongol candidates received easier questions and a quota for the number of successful candidates was instituted. This quota was the same for both the Mongols and the Han Chinese, despite the fact that the Mongol population was significantly smaller. As a result, the Yuan civil service exam produced few officials. One of the more important changes of the Yuan was that it made the Zhu Xi school of Neo-Confucianism the orthodox school and the tests revolved around Zhu Xi's teachings. This was adopted by both the Ming and the Qing.
The early Ming also did not hold the civil service exam, which Zhu Yuanzhang only begrudgingly reinstituted much later after all his other methods of recruiting officials turned out to be disappointing failures. By the late Ming, because of increasing commercialization and population growth, more and more people took the exam and this led to a huge explosion in the number of lower-degree holders. As a result, the Ming tightened its requirements and granted out less jinshi degrees. The Song throughout its history granted 39,711 jinshi degrees, the Ming only 24,536. But as with the Song, the process was usually dominated by wealthy families that already had a history of producing degree holders. The Qing basically inherited the Ming system without many modifications.
if there are any traces of it in contemporary Chinese society.
The concept itself survived. Today, there is a Chinese Civil Service Exam (中國公務員考試) that is held once a year for those who wish to enter the Chinese public service. The content, not so much. Confucian discourse and prose style is not relevant these days.