The short and simple answer is the profound impact of Romance of the Three Kingdoms. The long answer is that it was more complicated, and ever since the Eastern Jin there were already some people who saw Shu-Han as the legitimate successor to the Han.

As you stated, the Western Jin legitimized Cao Wei as the successor to the Han because it's own legitimacy was based on the ruler of Cao Wei abdicating in favor of the Sima clan. Therefore, if Cao Wei was not legitimate, then Jin was not legitimate. Furthermore, Cao Wei was the dominant power, controlled the heartland of China (including the two capitals), and received the ritual abdication of the Han emperor. By contrast, Shu-Han and Eastern Wu were both located in the uncivilized peripheries of the Chinese world. Therefore, Chen Shou's Record of the Three Kingdoms declared Cao Wei to be the legitimate successor to the Han and this view was the dominant view all the way to the Northern Song. Sima Guang's Comprehensive Mirror to Aid in Governance also painted Cao Wei as legitimate, and he used terms like 寇 (kou), which means to invade or plunder (but in a very negative sense, since the character is also used to describe bandits and enemies) when describing Shu-Han attacks on Wei. There is probably also a good political reason for this as well - the Northern Song also controlled the northern Chinese heartland but its inability to reconquer the Sixteen Prefectures meant it was unable to "restore" the territories of the Han and Tang. This meant its situation was actually very similar to that faced by Cao Wei, which was also unable to reunite China.

The Eastern Jin scholar Xi Zuochi (習鑿齒) is credited as the first historian to posit the "Shu-Han as legitimate" viewpoint. In his Annals of Han and Jin, which spans the 300 years from Emperor Guangwu of Eastern Han to Emperor Min of Western Jin (note that he deliberately skipped over Cao Wei), he argues that Cao Wei was illegitimate because:

• Cao Wei did not control all of China (the Han territories).

• Cao Wei rulers were not related by blood to the Han imperial house.

• Ritual abdication of the Han emperor was not enough to legitimize the Cao Wei ruling house.

Therefore, according to Xi, Cao Pi was simply a usurper in the same vein as Wang Mang. By contrast, Liu Bei as a descendant of the Han ruling house and flying the banner of Han restoration was the legitimate successor to the Han, and in his book Xi used Shu Han imperial titles and the Shu Han calendar. He argued that the Jin was the direct successor to the Han, not a successor to Cao Wei.

So the question becomes, why did Xi parrot this view? There are several reasons why:

• The Eastern Jin was confined to southern China (aka uncivilized territory, as northern China was still considered the heartland), therefore losing its claim to legitimacy if Chen Shou's theory is applied (remember that Cao Wei controlled northern China, therefore making it legitimate).

• Xi Zuochi was a huge fan of Zhuge Liang.

• Xi wrote the book as a warning to his patron Huan Wen, who was a Eastern Jin general who wanted to overthrow the Sima clan and become emperor himself. Xi wanted Huan to emulate Liu Bei as a righteous defender of the imperial house, rather than become a usurper like Cao Pi.

Xi's view, however, was still a minority view, and like I said above most people still subscribed to the theory of Cao Wei was the legitimate successor to the Han, including Sima Guang. But that changed during the Southern Song, with Zhu Xi himself coming out in support of Xi Zuochi's view. The Southern Song was in a similar situation, having lost control of northern China and becoming confined to the south. Therefore, using the viewpoint that legitimizes the dynasty in control of northern China would have meant the Jurchen Jin, not the Southern Song, was legitimate. Furthermore, Zhu Xi heavily emphasized the importance of blood relations, which meant only Liu Bei could claim the mantle of the Han dynasty.

So by the time Luo Guanzhong wrote Romance, the Shu-Han legitimacy view had already gained traction. Romance simply made it more widespread by painting Liu Bei as a humble, modest, and righteous figure, Zhuge Liang as a military mastermind, and Cao Cao as a evil villain. But as I've explained elsewhere in this subreddit a long time ago, most of what he wrote was historical BS, attributing things to Zhuge Liang that Zhuge Liang never did and turning both him and Liu Bei in overrated heroes. Luo was a huge fan of Zhuge Liang as well, and he undoubtedly read both Xi Zuochi and Zhu Xi's arguments on this matter and incorporated them into Romance.

I can't speak for the Ming dynasty attitudes but I can give some thought to the way the Cao claim to the mandate lost ground and Shu-Han's rose to add to the good answer given by u/lordtiandao. Some housekeeping first

Shu-Han's claim had nothing to do with the decree from Emperor Xian to Dong Cheng, that was more Liu Bei's justification for breaking away from Cao Cao and the events that followed. Shu-Han's justification was the usual omens, popular support, the qualities of the ruler but beyond the usual: the Cao's are cruel dirty usurpers who bullied their way to the throne and had no claim on it, the Han retains the loyalty of man after 400 years. Since Liu Bei is the last Han member standing in a position to restore the dynasty with Emperor Xian "dead" (that he wasn't dead wasn't a problem Shu-Han was overly concerned with). This was just another Wang Mang, a branch of the Han has been deposed by a wicked usurper and a Han relative, with Liu Bei being the new Emperor Guangwu, would rise to restore the Han.

Now in terms of Liu Bei's lineage, it was certainly distant but as far as the people of the time and those making arguments about the mandate were concerned, he was of the Han line. If we DNA tested him and discovered he was the secret love child of Cao Song and so Cao Cao's brother, it would be amusing but wouldn't shape how he was seen at the time. His tie was diplomatically useful with fellow Liu warlords, it also led to controversy over his marriage to Liu Mao's widow Wu because she had been married to a fellow Liu and in the end it allowed him to claim to be the successor to the long-lived Han dynasty.

The decline of Wei's mandate

Han and Jin, since both had their land under control, were both agreed upon legitimate dynasties. Even if the Sima's method of overthrowing the Cao's remained controversial and would haunt their reputation. The period between Emperor Xian's abdication and Cao Huan's abdication however there were three claimants and over time, two of them would be subject of debate.

Wu's never got off the ground as a claim, omens, popular support, Sun Quan's virtue made him the best since the Cao's were cruel and thus unworthy to rule and the Ha's mandate was long exhausted (his allies in Shu-Han were not best pleased) so he wins by default ... it lacked any real strong claim that the other two could call upon. Win by default, none of the traditional main cities of China, whatever impressive feats that Wu had wasn't going to win them the mandate.

For Jin, it had to be Wei as op noted. Cao Pi and his court had worked hard to legitimize the process of abdication, that a living Emperor could pass the mandate onto another family and Sima's ascension also required that. The Cao's claim needed to remain at least somewhat true, the Cao's were acknowledged as Emperors unlike their rivals and the SGZ carefully backs that legitimacy. The Cao's had unified the Central Plains, brought back authority to the government, great literary accomplishments so on and so forth.

The Sima's worked hard to portray themselves as protectors of the Cao family, their political opponents all traitors and ignore their claims to have the backing of the Dowager, that Dowager Guo had backed them and please ignore the Emperor we killed in the streets. Drawing upon the legitimacy.

Some dynasties like the Tang and Northern Song would support Wei, admiring Cao Cao for nearly uniting the land, for his military and administrative successes, drawing upon his successes to compare to their own.

However hostile works like Caoman Zhuan survived that would continue to damage the reputation of figures like Cao Cao, charges from Cao Cao's foes would remain remembered and help shape his reputation. Jin needed to justify their overthrowing the dynasty they had supported so, as well as advertising their virtue and merits that elevated them to such a position: questions of background compared to the Sima's Kingly descent, nods to the cruelty and eccentricities of the Cao's, the "disaster" of Cao Shuang's regime and association with philosophers like He Yan, that the Cao line had not been worthy enough, had wandered away from true Confucian values.

Over time tales spread of a crafty frivolous and sometimes brutal Cao Cao in later works like the Shishuo Xinyu. Even in the times of the Northern Song, popular plays were having Cao Cao losing to Liu Bei to the crowd's delight. He became often the crafty villain put out there be humiliated by Liu Bei and his companions while in 1316, Yuan official Shenyu Jiong destroyed a temple of Cao Cao, an act which became celebrated in poetry. Xiao Chang Xu Hou Han shu in 104 for the Song dynasty emphasis Cao Cao's brutality and that they took the throne by bloody forceful means (that Cao Cao had Cao Pi take the throne despite being dead should raise eyebrows)

It isn't great for your dynasty's claims to legitimacy when your most famous leader is the narrative bad guy while the fact that the Cao's had never won, despite the resources, raised questions about how legitimate they were and how proper it was that they tried to take the mandate.

Why Shu

So yes, Jin had no interest in Shu-Han being legitimate, Liu Bei and Liu Shan were First Sovereign and Second Sovereign, more honorifics than the Sun family got but not proper Emperor's, placed before their predecessors in Yi in the SGZ. Both Liu rulers had a reputation for kindness (Liu Shan's continued until the novel went and wrecked it), Zhuge Liang was worshipped in Yi moments after his death, figures like Guan Yu were famous but it was an illegitimate dynasty that had been swept away.

However, as u/lordtiandao points out, Eastern Jin historian Xi Zuochi began the process of "actually it went Han- continued on via the Han in Shu-Han then Jin via right of conquest." and over time that did eventually build. For some dynasties, the resistance against the bigger northern power was more appealing than claiming the big northern power was legitimate since they were a southern dynasty fighting the big northern power. Liu Bei, as continuing the Han line, was thus someone to emphasise for those seeking to themselves portray their dynasty as the legitimate continuation of past glory days fighting against usurpers or invaders from of the North. For some reflection from the long-serving Han dynasty then Shu-Han was also useful, sacrifice to Liu Bei and Liu Shan to be seen honouring the Han ancestors and reflect on yourself as Liu Yuan did in 304 as he founded his dynasty in 304.

While, for a long time, Wei's cause did trump Shu-Han's in court, the likes of Zuochi had left the question hanging with some taking firm decisions one way or the other and others not committing. Meanwhile, on the ground, Liu Bei's popularity was growing: His bonds with his friends became bonds of brothers as an idealized example of a sovereign subject relationship, Liu Bei's rise from peasant to Emperor while being connected to a long-lived successful dynasty appealed. Zhuge Liang turned from a worshipped administrator (so much so it was a headache for Liu Shan) into a sage strategist and loyal example of what a scholar should be, the worship of Guan Yu to appease his spirit in Jing became something more and religions began to co-opt him to win support against their rivals.

Plays had Liu Bei and his merry band humiliating the more powerful Cao Cao or celebrating Guan Yu's 1,000 li journey (or giving him a fictional child) were popular, temples were created for these heroes. By the time the Romance of the Three Kingdoms was created, it was building on centuries of plays and courtly argument that the Cao's were a wicked lot and Liu Bei represented an ideal example of a ruler's relationship, a man resorting the Han via his nobility and lineage.

The novel built upon what had come before but has also been hugely influential in how the era has been seen. Which, in terms of history, is greatly unfortunate given the novel is, though a great read, historically absolute balderdash. Perceptions of figures like Liu Bei get so skewed as either heavily mythologised into a man of virtue rather then a ruthless warlord or in trying to wrestle back against that perception he can get a bit of a kicking.