Get ready for angry bishops and dense theology.
The early history of Christianity is very murky, so it is hard to pin down a date for the start of the Catholic Church. This is partly because, for the first couple of centuries, Christianity was just another cult and very few people paid enough attention to write about it. It is also because the definition of what Christianity was changed greatly over the first three centuries; it went from a small Jewish splinter group with a focus on asceticism, sin and charity to a distinct, state-sponsored religion that built cathedrals lined with gold mosaic. The canon of scripture was up for extensive debate, with different bishops using different compilations of holy texts. But, generally speaking, the "Catholic" church came about in the 4th and 5th centuries.
Let's start with a basic historical outline of early Christianity.
According to Tacitus, one of the great ancient historians, Jesus was executed in in the reign of Tiberius. Some apostles abandoned the movement and went home, others continued to spread his teachings. They settled down in the cities of Rome, Antioch, Jerusalem, and Alexandria among others - we have the writings of some of these apostles preserved in the New Testament. There is then a gap - the next mention of Christianity was during the Great Fire of Rome, where there was some sentiment that this strange cult was responsible for the fire. Nero used this to divert away from rumours that he had started the fire and launched a persecution in the city. There is then an even bigger gap in the historical record. Christianity spread rather slowly, but it is though that by 300 AD around 10% of the Roman Empire were Christians. Then Constantine announced his belief in Christ at the Battle of Milvian Bridge in 312; according the church historian Eusebius, he saw the sign of the cross in the sky and was moved to put the Chi-Rho on the shields of his men. In 313 he issued the Edict of Milan, which granted complete freedom of religion throughout the Roman Empire.
At the time, Christians were bad for public order. There were rival bishops in many places, and too many different versions of doctrine to count. On occasion, this caused violence and clergy-led arson of rival churches. Many Christians also held a deep hatred of Jews, though this varied widely throughout the empire - in Antioch the Jews and Christians were on great terms and Christians were allowed to use the synagogues, but in Alexandria there were riots and pogroms against the Jewish communities. Christianity, despite now being endorsed by the emperor, was causing an enormous amount of trouble and as the emperor had become Christian, they all started appealing to him to sort things out. To try and understand what the hell was going on, and hopefully reach a solution, Constantine called a council of bishops.
The First Council of Nicaea in 325 declared the Nicene Creed, a bedrock of Christianity. Constantine had pressured the bishops into agreeing to it and it was a messy compromise between rival schismatics and theological parties, almost none of whom were genuinely happy with it. The faction which had formed around the Bishop of Rome developed the Old Roman Creed, later tweaked and renamed the Apostolic Creed, a feature of Catholicism and its western offshoots but not often used in the eastern churches. The Old Roman Creed was based on the writings of the second century writer Tertullian, and allegedly went right back the apostles but it is very hard to assess the validity of that claim. The first evidence we have of it comes from a letter to Julius, the Bishop of Rome as of 340, from an unnamed bishop angry at being forced out of his church by Arian clergy - one of the major rivals to the Latin church at the time. By 390, this was an established part of western church doctrine, as affirmed in a letter between Ambrose of Milan and the Bishop of Rome. This letter does not give a wording (the first wording of the creed to survive comes from Charlemagne), but it is clearly pretty close to the modern creed.
At this time the canon of The Bible was being formalised. There had been bibles before, but the inclusion of some books such as the Book of Esther, Wisdom of Solomon, some Psalms, and parts of Maccabees had been controversial. The modern Catholic scriptural canon was first laid out by Athanasius, Bishop of Alexandria, in 369. It was endorsed by Augustine, who oversaw many church councils which confirmed Athanasius’ canon and persuaded the Bishop of Rome to follow it. In 405, Pope Innocent I confirmed this canon in a letter to the bishop of Toulouse. The eastern branches of Christianity largely agreed with this, but over time came to diverge. For example, 3 Maccabees is not part of canon in western tradition, but it is scriptural canon in most others. Final dogmatic articulations of the canons were made at the Council of Trent of 1546 for Roman Catholicism to properly define it against the canon of Martin Luther.
In the middle of the 5th century, Christology became an intense and controversial issue. In case you are not aware, this was an argument over how divine and/or human Christ was. In 431, the First Council of Ephesus turned the disagreement into a major schism. The council was called over the views of the renegade bishop Nestorius. He believed that Mary was not the father of god, and by extension that Christ was part human, part divine, not both. The idea was that the council would come together, condemn him, and move on. Instead they wrangled over Christology. There were four factions: Miaphysitism (divine and human natures united) verses Dyophysitism (coexisting divinity and humanity, but not united) versus Monophysitism (only one nature, both human and divine) versus Nestorianism. The Bishop of Rome and his supporters were the Dyophysite faction, the Bishop of Alexandria led the Miaphysite faction, Nestorius was on his own, and the Monophysites did their own thing. The council endorsed the Bishop of Rome’s position, but it left bad feeling on all sides and the factions to become schisms. This was particularly bad because Dyophysitism was very, very close to Nestorianism, which they had come to condemn. The difference between the two strands of thought was mainly over the use of one term - “homoussian”.
The argument raged for decades, so in 451 another council was called - the Council of Chalcedon, also known as the Fourth Ecumenical Council. This reconfirmed the Dyophysite position and confirmed the primacy of Rome over all other bishops, creating the basis of the Papacy. The basis of the argument was Leo I’s Tome, which laid out his teachings and closely followed those of Augustine and Athanasius. Monophysitism and Miaphysitism were still very popular in the east, so this was seen as a slap in the face and the churches largely stopped working toward a common theology and would never be reunited. At this point, a recognisably Catholic church existed in the west, and there were many bickering churches in the east who didn’t talk to them. It was at this point that the Latin church started to be called Catholic, since the Fourth Ecumenical Council had declared their doctrines to be correct.
To summarise, although individuals such as Athanasius, Augustine, and Leo I were all important people, most of what defined the Catholic Church against the other churches came about as a result of bitter church politics, particularly over Christology, but by the middle of the 5th century, the western church was recognisably Catholic.
Sources:
Gallagher, Edmon L., Meade, John D. "The Biblical Canon Lists from Early Christianity: Texts and Analysis". Oxford UP, 2017.
Smith, Geoffrey S. "Guilt by Association: Heresy Catalogues in Early Christianity". Oxford Scholarship Online, 2014.
Rapp, Claudia. "Holy bishops in late antiquity: the nature of Christian leadership in an age of transition". Vol. 37. Univ of California Press, 2013.
Lee, Alan Douglas. "Pagans and Christians in late antiquity: a sourcebook". Routledge, 2015.
Price, Richard, and Michael Gaddis, eds. "The acts of the Council of Chalcedon". Vol. 45. Liverpool University Press, 2005.
Price, Richard, and Mary Whitby, eds. "Chalcedon in context: church councils 400-700". Vol. 1. Oxford University Press, 2011.
This is an incredibly tricky question, mostly because it's hard to define what exactly the starting point would be. The Catholic Church, for obvious reasons, would state that they began when Peter was conferred the keys to the kingdom of heaven by Christ. The Orthodox would say that they came into existence with the Schism of 1054, since they consider themselves the true protectors of the apostolic tradition. For the sake of clarity, I will be focusing on the most defining feature of the Church: the Pope.
The development of a single Bishop of Rome almost certainly did not happen within Peter's lifetime. In the early days of his ministry in Rome, the Christian community was most likely a series of small, independent house churches, not unlike systems of synagogues at the time. Not to mention how he was also there with Paul; Paul preached to Gentiles, while Peter mostly preached to Jews. However, in the following decades, it would condense into a loose hierarchy by the time of Clement; in his letter to the Corinthians in 96 AD, he refers to the Christians in Rome as a single, united Church. However, the notion of a single, clearly-defined leader was still vague. Tertullian of Carthage (155-240) wrote that Peter had ordained Clement as chief pastor of Rome before his execution in 64 AD, but didn't fully succeed the position until 91 AD. Meanwhile, two other men (Linus and Anacletus) were listed as Bishops of Rome in Irenaeus of Lyon's list of Popes compiled in the late second century. However, by the third century, Rome was regarded with a special religious preeminence due to not only being founded by two Apostles - Peter and Paul - but also being the location of their martyrdom. The sack of Jerusalem in 70 AD also hastened Rome's rise as the center of Christianity.
As time went on, Rome's preeminence became more concrete. Important bishops like Polycarp of Smyrna took care to remain in communion with Rome, meeting with Pope Anicetus in 155 to resolve issues concerning the date of the celebration of Easter. Rome was also awash in donations compared to other churches, so they took the responsibility of keeping the more impoverished communities funded. One of the first clear attempts by a bishop of Rome to exert his authority while citing the Petrine Succession would be the issue of Novatianism. After a wave of persecutions under Emperor Decius in the 250s AD, there was the question of what to do with Christians who had performed sacrifice to the Roman gods under duress, known as the lapsi. Several bishops, most radically being the first antipope Novatian, claimed that lapsi could not be readmitted to communion with the Church, and that lapsi priests could not perform the sacraments. Bishop Cyprian of Carthage took a more moderate stance, but still held that people baptised by a lapsi needed rebaptism. Pope Stephen I claimed that any Trinitarian baptism was valid, and that he had the authority to claim so through succession from Peter; this was also the first case that a Bishop of Rome cites Mt. 16:16-19 (the "you are Peter and upon this rock I will build my Church" verses) as justification for authority. Cyprian condemned Stephen for trying to proclaim himself a "bishop of bishops," but the issue was solved by the Romans when both Stephen and Cyprian were executed within a year of one another in a new wave of persecutions.
The tradition of Popes asserting their authority in more absolute terms became more prominent over the centuries. Pope Siricius (r. 384-399) was one of the first Popes to issue binding decretals in matters of doctrine outside of an official synod or ecumenical council. The Synod of Antioch in 341 had proclaimed Rome to be the 'capital of devotion' for all of time in the Christian faith and the Synod of Sardica in 342 gave bishops the right to appeal to Rome to resolve doctrinal disputes. St. Augustine famously wrote "Roma locuta est, causa finita est" (Rome has spoken, the matter has finished) when Pope Innocent I endorsed the outcome of two synods that condemned the heresy of Pelagianism. Pope Leo the Great asserted himself as maintaining a special bond from Peter by virtue of occupying the same position as him, and his letter to the Council of Chalcedon was praised with the proclamation, "Peter has spoken through Leo!" From then on, the idea of Papal primacy & Petrine succession was more or less accepted as fact within the Western Church.
You could technically say that Constantine did found the Catholic Church, since he did call the Council of Nicea, where the Bishops formulated what eventually became the Niceo-Constantinopolitan Creed recited at every Mass. However, he did little to actually dictate the outcome of the Council, and was more content that everyone was finally on the same page and that Christendom was united.