The Roman state was very powerful for its time, but compared to any modern nation-state it was an exceedingly weak one and not capable of silencing a group entirely. Many laws were issued by the emperors against pagans and heretics, yet it is clear that many of these groups thrived despite official 'persecution'; even into the mid-sixth century there were still pagan bureaucrats in Constantinople and the archaeological evidence suggests that the average Roman in rural areas continued to hold fairly heterodox and syncretistic beliefs. Our sources were overwhelmingly written by well-educated Christians living in the cities and their words obscure how diverse their world actually was. Christianisation was a gradual process, especially as Christianity itself was not a fixed concept. Constantine's conversion provided an impetus for people to try to properly define it, but no faction ever succeeded at being the one voice of their faith - throughout this period, doctrines were adopted and dropped, emperors rose and fell, cults grew and declined, so from my perspective no-one was ever in power long enough to enforce their will without strong opposition.

There are also other factors to consider. Between the fourth and seventh centuries, the period I know best, it seems like most emperors were happy to compromise with the many heretics in their empire and rarely resorted to mass persecution - it was after all pretty hard to clamp down on a group when the officials enforcing imperial edicts were vast outnumbered. This was true for example for the Donatists in fourth-century North Africa and the miaphysites in the Near East in the sixth/seventh centuries. I'm sure many emperors would have liked to rule a truly orthodox empire, which was after all what their propaganda announced to the world, but orthodoxy itself was negotiable in many cases and political realities often trumped personal principles.

Of course, many people did feel very strongly about certain issues and persecutions did occur, and many people were indeed willing to fight to the bitter end for their beliefs, but that did not mean that their opponents thought the same. Realistically, in every doctrinal dispute there were many people who fervently believed that they were right, but there were also many others who saw it as an opportunity to advance themselves or thought that the empire's resources could be better used elsewhere. In the seventh century for example a group of radical Chalcedonian monks (the 'orthodox' side) allied with the papacy and triggered revolts in North Africa and Italy against the emperor, who had embraced monotheletism, a 'heresy' for many people, yet for others it was seen as the natural development of orthodox Chalcedonian beliefs. The rebellious monks in fact held a council in Rome to condemn monotheletism, but anathematising the eastern patriarchs was as far as they were willing to go in ecclesiastical terms, since they wanted to reunite the Church by getting the others to come to their way of thinking rather than to split the Church. As I'm sure you know, persecuted groups tend to develop stronger identities and it would have been even more difficult to reunite with them later; the same applies to other examples throughout late antiquity as well, as people would only have look back at their recent history to see that persecutions just made everyone more obstinate and bitter. Anyway, the important point here is that as the monothelete controversy took place at the same time as the Arab Conquests, it is also clear that for many people these issues didn't matter that much, since the crumbling military situation in the east and the ensuing economic crisis were far more important for people in their daily lives. After the revolts were quashed, the emperor, Constans II, exiled the ringleaders but seemingly did little else, as he decided to focus on defending what remained of his empire instead; so much for the persecutor of orthodoxy we hear so much about in the later sources.

This also brings to mind another important point: many emperors and senior clergymen who were in the best place to trigger persecutions in fact held fairly unorthodox views, since as I said above there was no central authority defining what orthodoxy was. Constantine I for example dithered on the Arian issue and was eventually baptised by an Arian bishop, whilst his most successful son, Constantius II, was an outright Arian Christian of the Homoian variety and consistently bullied bishops to get them to follow his beliefs. Constantius was succeeded by Julian, a pagan, so Christian heresies weren't that big a deal for him. Jovian died too soon for us to know about his thoughts, whilst Valentinian I was a soldier-emperor more concerned about the military situation than religious debates. His brother Valens was a Homoian Arian, likewise for Valentinian's son, Valentinian II. The same goes for many emperors for centuries afterwards and indeed even for many clergymen - it is worth noting that many heretics were already bishops or patriarchs before their beliefs were condemned, so their theology already had an influence at the top of the Church hierarchy and they often maintained a strong following after their exile or death.

This only really covers the political side of things as I'm not so familiar with the theological justifications for de facto toleration, but hopefully someone else will comment on that. If you have any questions on what I have mentioned though, I'm happy to expand on them; there were a lot of heresies in this period and I really don't think my brief examples did them justice!