Roman religion isn't the focus of my studies although there is some correspondence between the emperor Trajan and his governor Pliny in the early 2nd century AD that covers exactly how the Romans felt about this issue. Although this example is specific to Christians, the thought process can be applied across other religious groups as well.

Pliny to Trajan,

in the case of those who were denounced to me as Christians, I have observed the following procedure: I interrogated these as to whether they were Christians; those who confessed I interrogated a second and a third time, threatening them with punishment; those who persisted I ordered executed. For I had no doubt that, whatever the nature of their creed, stubbornness and inflexible obstinacy surely deserve to be punished.

It is telling that the punishments are being enforced on account of the perpetrator's obstinacy, rather than any ruling against their belief system.

Pliny continues,

An anonymous document was published containing the names of many persons. Those who denied that they were or had been Christians, when they invoked the gods in words dictated by me, offered prayer with incense and wine to your image, which I had ordered to be brought for this purpose together with statues of the gods, and moreover cursed Christ--none of which those who are really Christians, it is said, can be forced to do--these I thought should be discharged. Others named by the informer declared that they were Christians, but then denied it, asserting that they had been but had ceased to be, some three years before, others many years, some as much as twenty-five years. They all worshipped your image and the statues of the gods, and cursed Christ.

As long as the accused followed the proper rituals they were released. They were not asked about their beliefs, that wasn't important to the authorities. What was important was the physical action and sacrifice to pay proper respect and reverence to their deities.

Finally Trajan himself echoes the point that, as long as an individual pays proper respects they are to be pardoned.

You observed proper procedure, my dear Pliny, in sifting the cases of those who had been denounced to you as Christians. For it is not possible to lay down any general rule to serve as a kind of fixed standard. They are not to be sought out; if they are denounced and proved guilty, they are to be punished, with this reservation, that whoever denies that he is a Christian and really proves it--that is, by worshiping our gods--even though he was under suspicion in the past, shall obtain pardon through repentance. But anonymously posted accusations ought to have no place in any prosecution. For this is both a dangerous kind of precedent and out of keeping with the spirit of our age.