An answer I gave to another question a while back could be useful to you. I hasten to add that "pagans" were not a monolithic group, and in some ways the distinction between pagans and Christians was very much born out of a Christian search for self-identity in the fourth century as Christianity begins to firmly take root within the structures of the Empire. Letters exist from pagan authors, primarily from the wealthiest landowners and senatorial classes, and there does not exist a strong sense of persecution from them. These families would often continue to practice their religion, minus the aspects of it that were banned as the fourth and fifth centuries went on (divination, animal sacrifices). Pagans still existed and formed a cohesive group within the capital by 410. However the continuing dominance of Christianity made it rather difficult for them to both participate in an increasingly Christian government, even if no one actively forced them to convert. This provided very strong incentives for these families to convert to Christianity that, it seems, few did not take by the late fifth century.
We should also not forget the large extant pagan or pagan-adjacent groups in the countryside that maintained their traditions, often alongside incorporating Christianity into their lives. The historical record on these peoples is very spotty, and taking what exists at face value is dubious at best. Bishops often complain about 'pagan' elements in Christian practices of the countryside well into the early medieval period; to what extent this was actual pagan practice rather than eclectic formulations of Christian worship of disconnected communities I'll leave to the experts.
If you were expecting stories of stake-burnings, inquisitional witch hunts and lynchings, it isn't that simple. Persecutions of non-Christians in the same manner in which Christians were once persecuted do not exist; there never was a systematic series of pagan executions The Roman conversion to Christianity was a long, drawn-out process that rarely involved effective systematic acts of violence enacted through decrees by Christian emperors. At most, laws were enacted to ban certain pagan practices, an increasing surfeit of limitations on pagan practices and their visibility in government. However this legal clamping down on paganism required a control over individuals that the Roman state never really had; just as with persecutions of Christians, the intensity was determined by the willingness of local communities to act on them. Sometimes emperors were forced to rein in the more zealous anti-pagan actors in order to preserve civic cohesion and public order. Influential pagans remained in government in the 4th century and beyond, temples remained in place in Rome, and cross the empire pockets of pagan worship and pagan traditions existed alongside Christianity.
The books I looked at for my previous answer:
A Companion to Religion in Late Antiquity (published by Wiley Blackwell)
Pagans and Christians in Late Antique Rome: Conflict, Competition, and Coexistence in the Fourth Century (Cambridge University Press)
From Rome to Byzantium AD 363 to 565: The Transformation of Ancient Rome (Edinburg University Press).