Christianity only became the official religion of the empire during the reign of Theodosius I (379-395). Before then, Christianity was favoured by most emperors but it was never imposed with any great rigour, certainly not enough to force all of the upper class to embrace Christianity, let alone the common people. As I discussed here, paganism was not swept away by Christianity. Having said that, Christianity did become the dominant faith in the fourth century, at least in the sources we still have. Even before Constantine, I think it would be reasonable to say that Christians were a large minority in the empire, big enough that it was worthwhile for emperors to favour them.

As I said in another comment here, Constantine's conversion was a gradual process, since he already had Christian advisors and relatives at his court, such as his mother Helena. His father similarly had protected Christians in his part of the empire, whilst emperors like Maxentius were happy to grant Christians the right to practice their religion (possibly only to annoy his enemies in the east, but it was still a degree of toleration at a time of persecution). Though the Later Roman Empire was autocratic, junior emperors were often happy to ignore the 'official' policy of the senior emperor and treat Christians leniently.

Even persecutions, such as the Great Persecution of Diocletian, are evidence of this, since they were not continuous nor were they always implemented with the same severity. They only occurred occasionally and under certain emperors. Lactantius, the Christian writer who later lionised Constantine, was for instance first invited to the imperial court by Diocletian to teach rhetoric - he resigned when Diocletian turned against Christianity, but clearly Christians were not excluded from the political world at this point, instead they were thoroughly integrated into it. Timothy Barnes recently also suggested that the Great Persecution was an attempt by the Caesar Galerius to influence imperial succession, as he knew that Constantine and Maxentius were Christian sympathisers, so by persuading Diocletian to do this, it effectively eliminated the two as credible successors to the Tetrarchy.

Constantine's choice of Christianity was also a small step from the traditions of previous emperors. Paul Stephenson for example listed many emperors before Constantine who had gradually drifted towards a form of monotheism, from the Severans all the way to Aurelian, most of them with a particular focus on Apollo/Unconquered Sun. It was thus I think a short leap of imagination for Constantine to go from believing that he was protected by a divine patron to believing that it was the Christian God protecting him, especially as he was always in contact with Christians. This process reached its conclusion around 312, but it was turned into a dramatic and world-shattering revelation by later Christian writers like Eusebius and Lactantius, who had every reason to say that Constantine embraced Christianity because of divine intervention, rather than because of a host of more mundane reasons. This isn't to say that Constantine's conversion was entirely pragmatic, but only that Constantine was living in a context where it was common to worship one god, so his conversion was not a break with his Roman context, but one very much in tune with it.

According to Roman sources, Constantine had a religious experience during the Battle of the Milvian Bridge, where he saw in the sky the Chi Rho (an early Christogram) and the words In hoc signo vinces which more or less translates to 'By this sign you will conquer'. He was purportedly baptised on his deathbed. Whether or not you believe that is up to you but that at least is what the Romans said happened.

Edit: The story I mentioned comes from Eusebius's Life of Constantine. Because of the writing style and Eusebius's relationship with Constantine you should probably take that story with a grain of salt.