The answer to your question is indeed long and somewhat complex, in any case we can made some key points that highlight the rise and decline of the papacy.

I think it is useful as a start to briefly describe the "original" (someone please correct me if in English religious studies other notations are used) christian community, that of the first centuries of the first millennia, particularly significant is the shift from a "Presbyterian" community (e.g: ruled by a "council of elders, typical of Jewish-eastern Christianity at that time) to the episcopal one (that is one with the bishop), where said bishop was a sort of warden, protector and sometimes ruler of the community.

Popes were, and still are, the bishops of Rome. In that they were exactly on the same level as any other bishop in the Empire (and sometimes beyond): in fact at least until the early middle ages the bishop of Rome was not the most important of the five patriarchy (the five most important episcopal seats of the world back then were Rome, Alexandria of Egypt, Antioch in Syria, Jerusalem and Constantinople after it was made capital of the empire) being ,as an example, less important than the Patriarch of Constantinople (since he was the bishop of the capital), as regarding the many and complex theological disputes that troubled Christianity - particularly during the 4th century - interested mainly the eastern part of the Empire (the first ecumenical Council held in the West was the Lateran one in 1123!), but was the only of the five to be in the western part of the empire and many western bishops (like in Spain and Gaul) used to follow the bishop of Rome "line".

The rise of the Papacy can be viewed then as a sort of struggle between the Bishop of Rome, who considered himself above all of the others, and both the secular powers of kings and Emperors and that of other Bishops of Christianity, a struggle that was fought with edicts, councils, fake documents (like the infamous Donation of Constantine), opportunistic alliances with new players on the European scene and many, many wars.

Besides being the bishop and Patriarch of Rome, with control over a large and rich urban community, at least until the Gothic war (6th century), almost all of the power the Pope came to have was either given to or usurped by the Papacy during its long history.

Regarding its acknowledgment as a secular entity (e.g. that has a temporal power over a territory), up until the Donation of Sutri in 728 (a second one in 743) Rome was formally (and actually) part of the Roman Empire so it held none of that power, being part of the Exarcate of Ravenna after the reconquest of the Italian Peninsula by Belisarius in the Gothic war (535-553). There was a Doux (or Duke, a governor) appointed by the Exarc in Ravenna (the actual then "capital of italy") that ruled over more, a Roman (or Byzantine if you prefer) garrison and so on. It withstood the laws of the rest of the Eastern empire and the pope was still "just" a Patriarch. Said donation was a return by the Lombard king Liutprand (the lombard had invaded italy after 568, shattering the Exarcate of Italy in many smaller pieces (among with ther was Rome). Subsequent wars between the Byzantines (or Romans) and Lombard were common, and during one of the treaties that were made, the Lombard king decided to return some strategic castles not to the rightfully owner (the Exarc and thus the Empire) but to the Pope (to be precise, the formula was "to apostles Peter and Paul", another common name for the holding of the papacy became since "the Property of Saint Peter"). thus recognizing him and not the Exarc. the Dukedom of Rome then declined and the Pope became a prime actor in Italy not just as a "spiritual" and theological power, but also as a temporal and actual power (as regarding the relationships with the Frank, or fighting the Arab threat in the Italian coasts).

Instrumental to this power are the many "barbarian" kingdoms looking for some form of legitimacy, thus turning to the Bishop of Rome - in a time where religious institutions were the only ones left in the west - as is the case of the Franks with Charlemagne, or the aforementioned Donation of Sutri by the Lombards king Liutprand. Important in this case is the role of the Pope in the creations of a "new" Holy Roman Empire. Here must be noted that in the crowning of the emperor there was not a passage of power from a titular rightful owner of the title, the pope, to the emperor, since the pope was somewhat lower to the emperor, similarly to the coronation of the British Monarchs by the Archbishop of Canterbury, where the Monarch is the head of the church, so was in some way the Emperor back then - but this is a more complex matter which requires a separate answer.

Suffice to say that the right of the pope to "create" kings and kingdoms was basically self given, as for the Normans who received a newly created Kingdom of Sicily in return for their service in defense of the anti-pope Anacletus against the Pope Innocenzo II and his supporters (among them France and Germany) in 1130.

Another key point is the compromise reached by the pope Callixtus II and the Holy Roman Emperor Henry V (the concordat of Worms in 1122), which for the firs time recognized a say from the pope in the election of Bishops (in the HRE) - until the 12th century bishops were elected by popular acclamation among the local clergy and recognized (or formally appointed) by nearby Bishops with the necessary approval of the local "secular" power, being a King or Emperor - often subject to local power struggles between important families or disturbances from local powers. Thanks to the Concordat of Worms the Pope now had the power to appoint Bishops (to be made in the presence of the emperor or a representative to receive the "temporal" power if outside of Italy, hence the strong power the Pope came to have in the peninsula), which basically made Bishops somewhat independent from Empires and Kingdoms and a sort of "agents" of the Pope, particularly where the secular power was weak and less where it was strong (like in France).

Regarding the figure of the Pope itself, it is important to note that he too was initially elected by popular (and clerical) acclamation like any other Bishop - some times appointed by the emperor (as with Otto II) and only later it begun to be elected by a "conclavis" of Cardinal-bishops (a figure which was created particularly for that reason) after Niccolò II in 1059. The elction of the pope (being either before or after the creation conclave) was in truth often a power struggle between Roman families which turned rome into a sort of battleground for much of the middle ages (many small castles and fortification erected by those families like Colonna or Caetani can be found around the Rome and its countryside) with the addiction of the occasional foreign power, which meant a Pope more often than not expression of a local (or foreign) particular interest: examples are the plethora of popes and anti-popes elected by different parties fighting between themselves (sometimes even three), ore the "captivity" in Avignon (1309-1376), where the pope was forcibly moved to Avignon (or actually he refused to come back to Rome since it was unmanageable and dangerous at the time) after it came under the strong influence of the French crown. All this undermined the papacy intent of being "universal" - since it often looked quite the opposite.

Before the Lutheran reformation (when these universal aspirations came across their end) I feel is important to talk about what i think is one the highest position the pope rose to have among the European kingdoms and the empires: the papal bull "Inter Caetera" issued by pope Alexander VI in 1493. In this bull (a sort of edict) the Pope formally divided the whole world (outside of Europe), between two kingdoms, Portugal and the united Monarchy of Castillia and Aragon, showing that at least as a formality there was a need to refer to the pope as a sort of "first among equals" at that time.

I hope i have answered at least partially to your answer, i'm sure i have forgotten some important things (and that i have written a couple of wrong ones), looking for someone to point that out and correct me!

p.s.: also, English is not my first language so i apologize for any error and for weird sounding sentences like this one.

p.p.s: about sources, this is a very superficial analysis that I have made, so basically any good university level textbook regarding European middle ages have these information. Also i am Italian so my sources are, naturally, Italians. What i have written here (beside the general history Textbook) is base on Santo Mazzarino's work on the late Empire and Late Antiquity I am not aware if there are any translation) and Manlio Simonetti's work on the early christianity.