This question can be controversial because people very invested in answering it are frequently heavily influenced by their own religious background in how they approach it (full disclosure: I'm Protestant). As a matter of historical scholarship, this is something that is difficult to know with any certainty because the sources we have don't address it very directly. Some of the only written sources we have for this period are the New Testament texts themselves, and how exactly we should ascribe dates to them is a topic of disagreement (many scholars believe the NT texts are of mid to late 1st century vintage, while others push them into the 2nd). We have some primary accounts from the Apostolic Fathers, the famous Shepherd of Hermas and texts purporting to be authored by Pope Clement I, but how we exegete these texts is often as much a matter of reading the text for what isn't there as much as for what is. Arguments from silence can be useful, but they are also easily abused since they can be used to support arguments that go in multiple directions.

What we do know is that the church in Rome had an association with St. Peter and his ministry from a very early date. The New Testament does not speak specifically to many of the things later attributed to Peter and Paul and their ministries in Rome, and much of this has been the province of tradition the evidence for which crops up later. While it is hard to know exactly what Peter thought of his own ministry, we do know that the church in Rome was thought of very highly by contemporaries, and was construed in some manner to be carrying on Peter's ministry. What exactly that meant has been the topic of disagreement. Some of our earlier texts - the Clementine Epistles, Hermas, and St. Irenaeus's Against Heresies - do have things to tell us about the nature of authority in the early church. The document traditionally called the First Epistle of Clement - the author does not identify himself - addresses Christians in Corinth. The document purports to be written on behalf of the church in Rome as a corporate entity. It does not appeal to the authority of a specific bishop, and while it does reference St. Peter, it is unclear how he was perceived in relationship to the ministry of the Roman church at that time. Hermas, likewise, makes similar statements about groups of ruling authorities within the church, but does not make categorical statements about the singular headship of a particular bishop in Rome. St. Irenaeus does make some strong claims about the nature of the Petrine ministry, but Against Heresies is a difficult text to interpret because our earliest complete copies are in Latin, and questions have been raised about how faithful the translation is in comparison with the original Greek texts of which only portions have survived. The evidence available at the moment has been interpreted by scholars as suggesting that a monarchical episcopate took some time to emerge in the early church, and that Rome may not have even had only one person exercising papal power for some time. There may simply have been no single pope. In the words of Eamon Duffy, himself a Roman Catholic: "[The church in Rome] was without any single dominant ruling officer, its elders or leaders sharing responsibility, but distributing tasks, like that of foreign corre-spondent. By the eve of the conversion of Constantine, there were more than two dozen of these religious community-centres or tituli." (Duffy, Saints and Sinners, p.11)

Popes were understood to carry out a kind of teaching office in the earliest centuries of the church, but what kind of teaching office that was became a topic of disagreement. Some argued that the occupant of Peter's seat was the conservator of tradition, and was bound by past decrees. On this understanding, one that has endured for centuries in some quarters, Popes were constrained by the tradition of the church. Others argued that the living ministry of Peter continued within the person of the present pontiff, leaving open the possibility of a declaratory judgment and development of doctrine. Even in the Greek East, where Roman primacy was always understood as being far more limited than in the West, Popes were understood to have a kind of appellate jurisdiction on certain matters of doctrine and church order. It was Christians in the East, for example, who appealed to Pope Leo I to nullify the decrees of the Second Council of Ephesus, sometimes called the Robber Synod. Over the centuries, as papal monarchy continued to develop, Roman divines articulated the primacy of the Apostolic See increasingly in terms of the Pope's power both to dispense judgment, and essentially to legislate, culminating in the First Vatican Council's dogmatic declaration of Papal Infallibility in 1870.

The short answer to your question is that we don't really know the particulars, since the sources do not speak plainly to us on these questions, but the best evidence suggests a long tradition attributing some kind of unique authority to Peter that started very early. What form it took is not entirely clear, and was probably articulated differently from how many Christians today would understand it.

Readings

Many of the more detailed histories of the papacy and the early church, unfortunately, are written in French or German, neither of which I read.

In English, one of the better comprehensive sources is Eamon Duffy's Saints and Sinners: A History of the Popes. As noted above, Duffy is a Roman Catholic, but not an especially partisan one in this book.

Also worth reading

Karl F. Morrison, Tradition and Authority in the Western Church: 300 - 1140

Kevin Madigan, Medieval Christianity: A New History

Geoffrey Barraclough, The Medieval Papacy

J.N.D. Kelly, Early Christian Doctrines

Timothy Ware, The Orthodox Church: An Introduction to Eastern Christianity. This text is by a Greek Orthodox bishop, and contains a summary of some Orthodox views of the papacy