There's no reason to think that Paul completely fabricated his meeting with Peter and other leaders, and ample reason to think that they did meet.  What exactly was discussed at that meeting, though, as well as who was there, when and why it was held, and how many times they met, is less clear.

To begin, let's look at the textual evidence.  In his letter to the Galatians, Paul first briefly says that he met and stayed with Peter (whom he calls “Cephas,” the Aramaic translation of “Peter”) for 15 days three years after Paul’s conversion, also saying that he met James, the brother of Jesus.  No other details of their discussions are given.  Then, in the second chapter, Paul describes two separate meetings, the first a private conversation in Jerusalem with Peter, James, and John, and the second a public disagreement in Antioch (all quotations from the New International Version):

Then after fourteen years, I went up again to Jerusalem, this time with Barnabas. I took Titus along also. I went in response to a revelation and, meeting privately with those esteemed as leaders, I presented to them the gospel that I preach among the Gentiles. I wanted to be sure I was not running and had not been running my race in vain… As for those who were held in high esteem—whatever they were makes no difference to me; God does not show favoritism—they added nothing to my message.  On the contrary, they recognized that I had been entrusted with the task of preaching the gospel to the uncircumcised, just as Peter had been to the circumcised. For God, who was at work in Peter as an apostle to the circumcised, was also at work in me as an apostle to the Gentiles. James, Cephas and John, those esteemed as pillars, gave me and Barnabas the right hand of fellowship when they recognized the grace given to me. They agreed that we should go to the Gentiles, and they to the circumcised. All they asked was that we should continue to remember the poor, the very thing I had been eager to do all along.

When Cephas came to Antioch, I opposed him to his face, because he stood condemned. For before certain men came from James, he used to eat with the Gentiles. But when they arrived, he began to draw back and separate himself from the Gentiles because he was afraid of those who belonged to the circumcision group. The other Jews joined him in his hypocrisy, so that by their hypocrisy even Barnabas was led astray. When I saw that they were not acting in line with the truth of the gospel, I said to Cephas in front of them all, “You are a Jew, yet you live like a Gentile and not like a Jew. How is it, then, that you force Gentiles to follow Jewish customs?” (Gal 2: 1-2, 6-14)

A meeting is also described in the book of Acts (Acts also briefly describes Barnabas introducing Paul to "the apostles" in Jerusalem shortly after Paul's conversion, but it does not say which apostles were present).  This seems to have been a public meeting in Jerusalem:

Certain people came down from Judea to Antioch and were teaching the believers: “Unless you are circumcised, according to the custom taught by Moses, you cannot be saved.” This brought Paul and Barnabas into sharp dispute and debate with them. So Paul and Barnabas were appointed, along with some other believers, to go up to Jerusalem to see the apostles and elders about this question. The church sent them on their way, and as they traveled through Phoenicia and Samaria, they told how the Gentiles had been converted. This news made all the believers very glad. When they came to Jerusalem, they were welcomed by the church and the apostles and elders, to whom they reported everything God had done through them. Then some of the believers who belonged to the party of the Pharisees stood up and said, “The Gentiles must be circumcised and required to keep the law of Moses.” The apostles and elders met to consider this question. 

After much discussion, Peter got up and addressed them: “Brothers, you know that some time ago God made a choice among you that the Gentiles might hear from my lips the message of the gospel and believe. God, who knows the heart, showed that he accepted them by giving the Holy Spirit to them, just as he did to us. He did not discriminate between us and them, for he purified their hearts by faith. Now then, why do you try to test God by putting on the necks of Gentiles a yoke that neither we nor our ancestors have been able to bear? No! We believe it is through the grace of our Lord Jesus that we are saved, just as they are.” The whole assembly became silent as they listened to Barnabas and Paul telling about the signs and wonders God had done among the Gentiles through them. (Acts 15:1-12)

After Barnabas and Paul speak, James gets up and quotes from the scriptures, then says that Gentiles should be welcomed without being circumcised.  The assembly drafts a letter saying as much and sends it back to the gentiles in Antioch.

So, Paul’s letter to the Galatians and the book of Acts both say that there was some disagreement in the early church over the question of whether Gentiles needed to follow Jewish law (especially circumcision), that Paul met with the Jerusalem leadership, and that the circumcision issue was discussed.  They disagree over most of the other details of the meeting.  Note that Acts describes only a single meeting, while Paul claims there were two.  Note also that both Peter and Paul’s roles are different in the sources.  In Galatians, Paul takes a prominent role by publicly rebuking Peter, unlike the lesser role he plays in Acts as something more like an emissary.  And in Acts, Peter takes the side of the Gentiles, while Paul claims he (later) sided with the pro-circumcision group.

But the fact that the sources disagree about the details of the meeting(s) is not evidence that no meeting took place.  In fact, it is actually evidence that some type of meeting did occur.  If the two sources agreed on all the details, it might be evidence that the author of Acts had read Galatians and was basing their narrative on Paul’s account.  As it is, it seems that Acts has gotten its information from a different source, relaying a different account of the meeting.  There is no evidence that the author of Acts had read the letter to the Galatians.  So, we have two independent sources claiming that Paul met with Peter over the question of circumcising gentiles.  It is also possible, of course, that both the Galatians account and the Acts account are more or less accurate, and they simply describe different meetings.  If that is the case, then there were at least three significant meetings (plus the time that Paul stayed with Peter).

Additionally, it makes logical sense that the two men would meet at some point.  Paul’s earliest letter (First Thessalonians) was probably written in the early 50s CE.  That’s only 20 years after the crucifixion.  Even if we throw out the Acts narrative as unreliable, Paul certainly had been active as a missionary for some time before writing to the Thessalonians.  Additionally, he was from Tarsus, in the Eastern Mediterranean, and had been educated in Jerusalem.  It stands to reason that a prominent missionary active only 15-20 years after the beginning of Christianity (and perhaps earlier) would meet with the early leadership from Jerusalem.

That being said, Paul may very well have “played up” his role in the early disagreements over circumcision.  His letter to the Galatians is specifically about this disagreement, apparently still ongoing.  It seems that certain people had come to Galatia and begun urging the Gentile Christians there to be circumcised.  Paul wrote to the Galatians to tell them not to do so.  He describes his meeting(s) with Peter and the others as a rhetorical device to strengthen his argument.  Paul claims to have publicly disagreed with Peter.  Who could do that but an equal?  And if Paul is important enough to publicly rebuke the most prominent of Jesus’ own disciples, then who are these nameless agitators who advocate for circumcision?  Certainly not anyone authoritative enough to go toe to toe with Paul.  (Note that I am not saying that the Antioch meeting absolutely did not happen in the way Paul describes - I’m saying we don’t know whether it did or not.  It does not line up with the narrative in Acts, which is otherwise very pro-Paul.  So either the meeting didn’t happen in this way, or the author of Acts was not aware of it, or else they chose not to include it in the finished text because it wouldn’t do to have Peter advocating circumcision in a book that is written for gentiles).

So, did Paul meet with Peter and the other apostles?  Almost certainly yes.  What did they talk about?  Probably circumcision.  How contentious was this meeting, and how many meetings were there?  We don’t know.