Thing is the bishop of Rome wasn't necessarily the most powerful patriarch of Christiendom : it is true that it claimed some primacy over other main seats of Christianity as the bishops of Rome were considered the successors of Peter from one hand, and ruling from one of the most prestigious imperial centres, but nobody really agreed on what this primacy was or how it was to be applied.While Leo I, supported by the western emperor Valentinian III, asserted Roman claims to primacy in 451 during the Council of Chalcedonia, it was made clear too that the Patriarchate of Constantinople would enjoy a similar primacy especially in the East.

And, in facts, the Roman Papacy only enjoyed a symbolic (if pretty much real) dominance over western metropolites : Milan and Ravenne beneficied from a closer relationship with western emperors and patricians such as Ricimer (and actually tended to have not a good relation with them). Roman influence over eastern churches policies was superficial at the very best. As Papacy fall under patrician/Barbarian control over the late Vth and VIth century, its prestige was fragilized.

Constantinople, on the other hand, was closer to imperial authority than any other seat, and thanks to this could arguably actually enforce its primacy over the other eastern patriarchates of Antioch, Jerusalem and Alexandria, following a broad imperial religious policy, and Constantinople could really be considered the most powerful patriarchate of western Christiendom during Late Antiquity.

The Vth century, while Roman pontiffs tried to assert the independence of the Church to the political power, was essentially a small "Dark Age" : most of royal authority in the region was Homoian (even if, at the notable exception of Vandals, not actively anti-Niceans) and Nicean Barbarians as Franks directly ruled ecclesiastical matters with their clergy, as Roman popes were, if autonomous, essentially set within Ostrogothic political influence. Seats like Carthage, without being really recognized as patriarchates, tended to function largely autonomously.

Byzantine reconquest of Italy, while putting Rome into Constantinople's sphere as much it was done in history which wasn't without real resistence and hostility in Italy itself, especially as Justinian proclaimed Constantinople the "chief of all churches", but it contributed to remove most of simony in favour of imperial nomination and to connect Rome with the general Christian theological community.

With the fall of Ostrogothic Italy, Nicean Merovingians took the lead in Western Europe where what remained of Homoian Christianity quickly vanished and began to be a political and religious influence for pagan kingdoms-in-building (such as Kent, Sussex and East-Anglia), while inner dynamics and imperial policies led to the adoption of the Nicean Credo by Spanish Goths.

In the same time, Rome and the territory it overseed, became a haven for clergy fleeing Lombard, Slavic and Persian armies (and later, Arabs), effectively making the patriarch not only more connected with the theological problems, mostly christology and defence of Chalcedonian Credo.As Latin Christianity grew, ecclesiastical assemblies and organisations whuch were still first and firstmost managed trough royal authority, acknowledged a significant spiritual and pastoral leadership from Papacy.

Imperial religious policies, critically the Three Chapters, monoenergism and monothelism, tended to be ill-recieved outside Constantinople, and the clergy of other patriarchates (especially Jerusalem, due to a lot of monastic ties with Rome since the Vth century) which opposed imperial policies looked to Rome for championing their struggle, which Rome did, less to increase its overall authority but maybe more because there wasn't much motivation for them to compromise over Christology as the matter was resolved in the West.It might seems bizarre as Roman Papacy was largely under Imperial thumb in the VIIth century. But thanks to the apparent triumph of Nicean Credo in the west and the conversion of England, a growing political autonomy born out of the Lombard advance in Italy and emperors being busy with Persians, Roman pontiffs were able to lead a short schism over monothelism, further increasing their prestige in Christiendom.It would be wrong, however, to consider the primacy of Rome established : even in Italy, pontifical authority was challenged during the Three-Chapter controversy, as pointed by the schism of the self-proclaimed Patriarchate of Aquilea. Both the emperor and the patriarchate of Constantinople readily acted against Rome there, intervening in Roman matters and putting popes in inner exile at Constantinople.

Similarily, it's not to mean the roman partiarchate was crushed either : the emperors wanted to keep a working relationship there, rather than burning bridges (especially as Constans II toyed with the idea of moving the center of the Empire in Italy). The continuous stream of Greek and Syrian popes of the late VIIth were rather a compromise with Latins than any political imposition. In fact, Latin Church, under Rome, became more and more autonomous and (partly because of these non-Italian popes) became to identify Roman church with Christianity.At this point, indeed, Latin Christianity's pastoral and ritual differences with Greek and Eastern Christianities became obvious, and Latins simply didn't considered abandoning it, considering these to be regular and catholic.So, when Justinian II attempted to backtrack on the compromise and impose Constantinople's considerations and Greek rites, backleash was immediate in Italy (while the clergy in Rome was mainly Greek and Syrian), and the emperor adopted more conciliatory tone which was met by Roman church without too much trouble : the main focus of both emperor and pontiff was to keep a Roman and Christian commonwealth alive.

Nevertheless, what really killed it was the adoption of Iconoclasm by Leo III. Rome was really keen to championing against imperial "innovations", just as before. This time, especially due to the capture of the other patriarchates by Arabs during the century, the opposition was dualistic rather than polycentric, and Leo III reneged on most of the compromises made by his predecessors, which achieved to alienate him the Roman church.Eventually, as Lombards tried to benefit from Roman isolation to achieve their conquest of Italy, Stephen II turned to the main power of Western Europe, Francia that Arnulfids (also known as Peppinids, later named Carolingians) reached a large prestige having defeated Arabo-Berbers during the 730's, while having built a tight relationship with Latin clergy with several missions in Frankish Germania. This forged a relation which climaxed with the sacrality of Carolingians and the establishment of the Carolingian Empire, as an alternative Christian Empire compared to Byzantine (the title of Charlemagne "emperor ruling over the Empire of Romans" is rather a reference to the pope, incarnating the will of Romans, than a reference to the old WRE).

While Frankish Papacy, at least initially, was really under the thumb of Charlemagne, it took more and more autonomy as the sole religious reference in the West, even as Byzantine Emperors recanted iconoclasm. The late IXth and early Xth was particularily messy for Rome, which lost a lot of prestige in the anarchy that followed the fall of Carolingia; but reforms of the Xth century essentially strived to restaure Roman sole primacy, not just in the West, but also universal primacy as Constantinople obviously needed some theological discipine as far as Latin Christianity was concerned.

Basically, the primacy of Roman church was always more or less asserted by Rome, but didn't really existed in practice for a long time. It was concretly built over a shaky relationship with imperial Christianity, the triumph of Nicean Credo in the West, and a defiance to imperial religious policies. It wasn't really ever acknowledged by Constantinople, but this is what made the claim of universal primacy a reality in the West at first : there are later movements (Ottonian reforms, political decline of Byzantine Empire, etc.) which reinforced it, of course, but they were mostly built over this evolution.