Constantine
Constantine the Great didn't officially convert to Christianity until 337, immediately before his death. In 312, he passed the Edict of Milan, which de-criminalised Christianity, but didn't establish it as the state religion and didn't ban paganism. Constantine did make his Christian preferences clear, and described polytheism as an "outmoded illusion." Eusebius, a contemporary Christian scholar, describes Constantine's actions against paganism in Vita Constantine:
... [Constantine] sent governors to the several provinces, mostly such as were devoted to the saving faith; and if any appeared inclined to adhere to Gentile worship, he forbade them to offer sacrifice.
and:
... he used every means to rebuke the superstitious errors of the heathen. Hence the entrances of their temples in the several cities were left exposed to the weather, being stripped of their doors at his command; the tiling of others was removed, and their roofs destroyed.
And so on.
How trustworthy in Eusebius? We don't really know. He was a devoted Christian and it reads very obviously in his works, and it's likely he embellished Constantine to promote the Christian cause. Modern scholars argue that Constantine opposed paganism, but didn't have the means to take major actions against pagans during his lifetime.
Constantius II
Between 337 - 361 Constantius II reigned as emperor. Constantius II was opposed to paganism, and slowly ramped up his anti-pagan campaign throughout his reign. He began by banning the practice of sacrifice, moved on to shuttering temples and other pagan places of worship, and eventually established the death penalty for anyone caught practicing pagan sacrifice and idol worship in 353.
Even so, Constantius II did not ban paganism outright and did not forcibly convert pagans. Paganism was still a powerful force in the empire, and it is likely that Constantius' decrees were not strongly enforced.
However, the state became increasingly dominated by Christians, which laid the groundwork for Theodosius in the 380s.
Libanius describes the widespread destruction of pagan temples during this period, which he lamented, as he felt the temples could easily be converted to Christian use. Libanius was a pagan Greek who likely opposed the persecution of non-Christians, but he was also a respected member of the Imperial court.
The emperors succeeding Constantius II were much more tolerant of paganism.
Theodosius I
Christianity wouldn't become the state religion of Rome until 380 with the Edict of Thessalonica. Between 389 and 391, Theodosius I passed laws effectively banning paganism in the empire. Theodosius I also decreed the destruction of pagan temples in Alexandria, including the Serapeum, which may have housed a section of the Library of Alexandria.
Theodosius was the first emperor to outlaw paganism and seek the forcible conversion of pagans to the state religion.
Sources:
Eusebius Vita Constantine [Online]
Libanius Oration 30 [Online]
Codex Theodosianus