This will be a copy-paste job from a thread I did on BH once. I think it will adequately address some of the subtleties of the question and why it's not perfectly straightforward.
R5: Comically oversimplified, gives a terribly bad impression of the history of the Papacy. This is not my area of expertise and I fully expect (and welcome) corrections from the rest of BH.
As a religious topic, a frank exchange of views may be offensive to some. Let me clarify that if you believe that the history of the Church has played out the way it has because of God's will and that modern Catholic doctrine is divinely inspired I have no problem with that even if I do not believe the same. My aim here is to discuss the development of the Papacy, not to tar doctrines as "untrue" or "wrong."
Okay, so it's a single line with a list of names and a handful of major famous events. There's not a whole lot of place for detail here. However, the naive reader would be terribly misguided if they were to only read this chart. Seriously, I've seen catechism pamphlets that give a better idea.
I'll give a very sketchy overview of two examples of badhistory by omission here - first, the idea that the Bishop of Rome has been comparable to the modern Pope from the very first; second, that there has been an unbroken, incontrovertible line of succession from St. Peter to Pope Francis.
The Papacy as we understand it did not appear fully formed, but developed over time and continues to evolve
If there's three things that distinguish the Bishop of Rome from other Bishops in the Church, those are
Papal Supremacy
The doctrine that the Bishop of Rome has authority over the rest of the Bishops. While the modern Church holds that bishops are appointed by the authority of the Pope, this is quite a modern practice. The earliest bishops were chosen by the faithful, by the priests over whom he'd have authority, by secular rulers or simply chosen by the bishop they'd replace. The Investiture Controversy wasn't fully settled until the middle of the 19th Century, which event probably had more to do with secular authorities generally adopting a separation of Church and State rather than the Papacy imposing itself.
The doctrine of Papal Supremacy is canonically based on a reading of Matthew 16:17-19. However, the earliest Bishops of Rome seem to have made no such claim, and authority over the Nicene Church (to which the Bishops of Rome belonged pre-schism) was held by a mix of clerical and temporal powers, importantly the Byzantine Emperors. From 537 to 752 the Bishops of Rome depended on Byzantine Imperial approval for episcopal consecration (selection of bishops) even for parishes directly dependent on the Pope. Constantius II went as far as to exile Pope Liberius (ruled 352-356) from Rome, as happened again to Martin I in 653 and several others. Damasus I (ruled 356 to 384) was named "Bishop of Bishops" by the Emperor, which may speak to the authority of the Bishop of Rome, but also speaks to the authority of the Emperor over him. The first claimant of Papal supremacy as we understand it seems to have been Innocent I (ruled 401-417), but this was not generally accepted within the Church. From 476 Rome was generally ruled by so-called heretics, occasionally occupied by Byzantine troops who had no interest in undermining Imperial authority, and overshadowed by more important parishes within the Church. Only in 616 did a Catholic inherit the throne to the Kingdom of Lombardy, and the struggle with the Emperors for supremacy within the Church would continue until the Great Schism in 1054.
This is not to say that the Bishops of Rome did not hold a position of honor among other Bishops. The point here is that for about half of Church history, the Pope was not the boss.
Papal Infallibility
The doctrine that the Pope may speak infallibly on matters of doctrine. As I showed above, the doctrine of Papal Supremacy developed over a centuries long period. Papal Infallibility did not become official Catholic doctrine until 1870 (First Vatican Council), but this was an acknowledgement of the status quo rather than a new idea. The earliest example of someone using the idea of Papal infallibility as an argument to settle a theological dispute may be the case of Peter Olivi in the 13th Century, but this was actually rejected by the Pope John XXII. Nevertheless, the idea that the Pope would be the final arbiter of doctrine in the Church gained ground, enough that several Catholic theologians wrote against the idea before Vatican I finally settled the matter.
Papal Sovereignty
Uniquely among (current) bishops, the Pope is the Head of State of a sovereign nation (the Vatican). This is not a new thing: While the modern Vatican State was created by a decree issued by Benito Mussolini in 1929, the first independent Papal State emerged (more by custom than anything else) sometime in the 7th Century as the Duchy of Rome, and continued until the Papal States were finally lost to Italy in 1870.
Nevertheless, we have a period of some 600 years between the birth of the Church and the creation (if that's even the right word) of the Duchy of Rome when the Pope was in no sense a temporal ruler.
As a related question, when the Popes were under Arian Lombard kings, were the Popes Arian or (proto-)Catholic?