It's a great question without a definitive answer, as the historical record has given scholars ammunition for competing theories over the centuries. The genesis of this uncertainty is the fact that dynastic historians, bound to conceptions of a "mandate of heaven" that passed successively between deserving clans, generally upheld both dynasties' self-legitimizing claims of Han patrilineal descent. And, as you have pointed out, there have been plenty of xenophobic and anti-Chinese texts over the years that have not presented the facts in a scholarly context. Ultimately, there has been recent scholarship that speaks to this point in a much more neutral vein.

It's important to note (as the historian Sanping Chen points out in his research) that both the Sui and Tang were seeking to lead a unified and stable China at a time when, for centuries, the Han-ruled southern realms had successfully defended their independence from the clearly non-Han northern dynasties that the Sui and Tang succeeded. Chen provides strong evidence that, despite gradual processes that blurred ethnic lines to varying degrees within the political systems of given dynasties, ethnic tensions between Han and non-Han cultures were quite palpable in this era (he notes, for instance, an instance of ethnic cleansing when non-Han forces sacked a Han-ruled southern city in the 550s, just decades before the rise of Sui). So for both dynasties, which emerged out of the political ethos of the northern dynasties, an accepted claim of Han patrilineal lineage would have eased the process of gaining acceptance with the southern elite and populace.

For what it's worth, the most pertinent modern scholarship on this topic has focused on the Tang. The historian Chen Yinke, for instance, argued very firmly that the Tang imperial family was of Han patrilineal descent and accepted the dynasty's claim of descent from the prominent Longxi Li clan of the Han dynasty. In recent decades, the scholar Sanping Chen has done some in-depth work on the Tang lineage and concludes that the Li clan were, rather, of Xianbei patrilineal descent. His case regarding the Tang imperial house is largely circumstantial, relying on (compellingly presented) evidence showing that through the reign of Emperor Xuanzong in the mid-700s, the Tang imperial clan followed distinctly steppe traditions in their personal and political lives. Many of these customs directly conflicted with the Confucian mores urged on the imperial family by the Han-ethnicity scholar-bureaucrat class, including a tumultuous Tang succession custom that, not unlike steppe succession customs, accepted bloody interfamilial feuding as a means of allowing the savviest and most capable heir to rise to power. Chen also presents evidence (secured through genetic testing) that another Li clan of the Sui-Tang era, which also claimed descent from the Longxi Li clan, was of unmistakably Xianbei descent.

Perhaps most interestingly, Chen points out that on at least two occasions - once during the reign of Emperor Taizong of Tang, and once during the Song dynasty - scholar-officials directly questioned the Li clan's origins. Taizong pressured the scholar who challenged him into backing down, although he treated the man quite leniently and, in doing so, made references acknowledging that the clan's origins were not as clear as the official records suggested. Chen's takeaway from this incident is that the obscure (and, in his view, likely Xianbei) origins of the Tang family were generally understood and acknowledged at the time, including by the Li clan itself, but that the empire's elite nevertheless consciously accepted the "fictitious" origin story as a means of securing dynastic legitimation that in turn stabilized the political order after a long period of disunion.