This is a difficult question to answer with complete certainty - it's a matter of serious and long-running scholarly debate, and has the potential to raise heated passions on either side, especially when people find it hard to detach personal feelings from the matter. The evidence is hard to piece together, which certainly doesn't help the situation. I am not an expert on Christianity, so I can't really get into the detail of how Christmas came to be fixed on 25th December, but I can talk about the cult of Sol Invictus in Rome. I wrote part of my Masters dissertation on this, so it's nice to get back into it! Others may come and fill in the other side, and you can also check out the Catholic Encyclopedia's entry for Christmas, which is impressively detailed and even touches on the possibilities of pagan borrowings.
So, onto the cult of Sol Invictus. The Sun had been worshipped in Rome from very early on, with Varro (5.74) claiming that Sol and Luna (the moon) were among the gods introduced to the city by Titus Tatius - the Sabine king who briefly ruled alongside Romulus. Sol appears on some early Roman coins, and seems to have had two temples in the city - one on the Quirinal hill (Quintilian 1.7.12) and another in the Circus Maximus (Tacitus Annals 15.74). He was a relatively minor deity in the Roman pantheon, but was nevertheless integral. In the early imperial calendars (none dated later than the early 1st century AD) there are two festivals for Sol - 8th or 9th August at the Quirinal temple and 11th December. There also seems to have been a festival for Sol and Luna together on 28th August. There were certainly priests of Sol in the city, as attested by epigraphic evidence, but the position seems to have been fairly lowly, perhaps restricted to freedmen and perhaps equestrians (see Rupke's epic Fasti Sacerdotum, which catalogues every religious official in the city of Rome between 300 BC and AD 499. It's a seriously impressive piece of scholarship, and does not get anywhere near the recognition it deserves).
The problem we have next, and with tracing all Roman religious festivals in the imperial period, is that after these early imperial calendars we have very few others until the Codex Calendar of 354: a significant gap of at least 300 years. It is this calendar that is the first to mention the Natalis Invicti (literally "Birth of the Unconquered One") on 25th of December, which means that the festival must have been instituted at some point in that 300-year gap. Of the festivals listed on the earlier calendars, only the one for Sol and Luna on 28th August survives to 354, and, along with the 25th December, there are also new festivals on 19th and 22nd of October.
The main contender for the institution of these new festivals is the emperor Aurelian, who ruled 270-275, and who elevated the cult of Sol after his victory of Zenobia of Palmyra. The story goes that he dedicated his victory to the Sun god, and on returning to Rome built a magnificent new temple to Sol on the Campus Martius and instituted a new festival in his patron god's honour. We don't know exactly where the temple was, but the best guess puts it under the Church of San Silvestro in Capite. Hijmans, who has done considerable work recently on revising our understanding of Roman Sun worship, has pointed out that, although the calendar of 354 lists 30 chariot races for the festival on 25th December, there are an unprecedented 36 listed for the other new festival for Sol on 22nd October. Hijmans thinks that this must be Aurelian's festival, which would have been intentionally showy - a brash stamp of imperial authority on the religious life of the city.
So, if 25th December isn't Aurelian's new festival for Sol, what is it? As I've said, there's no evidence for a Roman Solar festival on this date before the calendar of 354, which means it must be a fairly late addition. There was a small but influential clique of prominent pagans in 4th-century Rome - men like Vettius Agorius Praetextatus and Quintus Aurelius Symmachus - who openly fought for what they saw as traditional Roman religion. Perhaps the new festival was instituted under their direction? They dominated the position of Urban Prefect for a good portion of the mid-4th century, and so would certainly have the power to do so, and we know they were wealthy enough to afford it. Whether it was them or not, it would certainly be possible to argue that it was done in response to the institution of 25th December as Christmas day - a new pagan festival to compete with or to borrow from the Christian celebration.
As I said, I'm not really qualified to present the evidence for the Christian debates over the date of Christmas - hopefully someone else will fill in that side of this debate. If you want to read more about Sun worship in Rome I cannot recommend Hijmans's work highly enough. He specifically addresses the dating of Christmas in this article, and his PhD thesis is still open access online (here), hidden away in the bowels of the University of Groningen's archive. On the Calendar of 354, itself an incredibly interesting document, see M. Salzman, On Roman Time.