To expand on the question, wasn't the Catholic Church also considered to be one of the major heirs to the legacy of Rome?

There are two ways to look at this.

One is to assume that only the direct descendants of the Roman system, the Byzantines (who called themselves Roman) are the true heirs to the Roman Empire. Indeed, they wouldn't even consider themselves heirs, and most historians wouldn't either. Rather, they were a continuation of the same polity and structure of the Roman Empire, including its laws, traditions, and form of rule. In this sense, all the others are merely 'pretenders'.

The other way is to recognize that the 'heirs' of the Empire in the West, namely the Roman church and the Franks (and earlier, Theodoric's kingdom) were true heirs, in that they inherited and utilized Roman institutions such as the Church and administrators, and created new polities out of them. In this sense, they were not reclaiming or restoring the Empire, but rather, were true heirs to the history of that Empire in the territories they commanded.

I think both of these ideas have kernels of truth in them. There is no doubt that many polities and leaders, ranging from Charlemange to Mehmed II to the Tsars of Russia, attempted to claim the mantle as 'successors' to Rome. In Mehmed's case, he simply attempted to paint the Ottoman conquest of Constantinople as a dynasty change, and that the new Ottoman Empire was simply another form of the Roman Empire continued (and thus a true heir). Ultimately, these were all political attempts to justify expansion and power, and support the interpretation of the first idea above. On the other hand, the fact the Frankish Kingdom, and even the Ottomans, could utilize many of the government structures of the former Empire, and the fact that the law codes of so many former states owe their existence to the Roman model, should not be forgotten. In this sense, most of the territory that Rome controlled or influenced, ranging from the Peloponnese to the mouth of the Rhine, has vestigial influences remaining from Rome that makes them heirs to its tradition in one way or another. At what point does the synergy with the new cultural attitudes from Germanic, Arabic, or Turkish migrations, cause that status to erode beyond recognition in these former territories of Rome? Does the survival in part of Greek philosophical tradition in the Arab Caliphate count as being a heir? Does the legal tradition adopted from Justinian's Code count?

Ultimately, this question is answered by what one's personal definition of heir means. If one believes that only a direct continuation of the Roman state counts, then the Byzantines are you answer. If one takes a more nuanced view and includes the lasting cultural, lingustic, and legal legacy of Rome on its former territory (and beyond), the answer suddenly becomes a lot more broad. Even Moscow, with its pretensions to be a 'Third Rome' has some argument to be made there vis a vis its continued home of the largest Orthodox community. Is this a continuation of Rome or a continuation of Byzantium? It starts to get really complicated! As it should be.

The Islamic world isn't an heir to the Roman empire, their cultures were massively different. They possessed the same lands, yes, but Roman and Arab empires were massively different. I dont think the Arab empires had much of a basis in Rome. They followed their own path- we inherited stuff from them, like our numeric system.

Of the other two, both deserve the title in two seperate areas.

The 'barbarian' kingdoms evolved directly into our modern day European states. Most directly, the Kingdom of the Franks, and the effective successor state, the Holy Roman Empire (which grew out of hald of Charlamagnes conquests, who was a Frank) are the most befitting of the title 'heirs of the Roman Empire'. It's about equal, the Franks inherited more Latin in their language than the Germans afaik, but the Holy Roman Empire controlled Rome/operated as an empire more than the Franks.

Funnily enough, the 'eagles' of Legions was part of European armies long after Rome was gone. Napoleon's Grand Army had them, imitating Rome, as did the Russians (who took up the idea following the Byzantine Empires collapse)

The initial barbarian states do not qualify, it is their descendants that do. Hence, Britain's celts don't qualify, as it took until about the time of Alfred the Great. The barbarians further west were more Slavic than Roman.

And the Byzantine Empire was known as the Eastern Roman Empire for an extended period of time. It went on to control parts of Italy and even Tunisia I think at it's greatest. Even though Greek became the official language, in the royal court, they stuck with Latin. Even before the fall of the Western Empire, they had become a significantly different culture, which Greek influences more predominant than in the West. They were also the bastion of knowledge and civilization during the Dark Ages, moreso than the Church.

So, in order, Byzantines -> Barbarian successor states -> Islamic states

All three are heirs to Rome in different ways. It doesn't make much sense to try to quantify that and say one is the most.