This is tricky. Part of the problem is that "religion" and "atheism" didn't really mean the same thing to Romans as it does to us. You couldn't ask a Roman "What religion are you?" and have him answer "Oh, I guess I'm a lapsed pagan. I used to go to all the sacrifices and festivals regularly, but I don't really believe in any of that any more, so I stopped going."

Deities were concrete objects first, who also had domains of power in abstract concepts. So, Venus was the goddess of love (an abstract concept), but also sex (not really an object, but a physical act, the existence of which can't really be ignored or denied). Here's an analogy: Say you and I are both in the same room, and you notice me praying to a chair. You ask me what I'm doing, and I say "I'm praying to the chair god so the chair won't break when I sit in it. I do it every day." And you ask, "You don't really believe in a chair god, do you?" And I say "I don't understand. The chair's right there, isn't it?" Neither of us can deny that the chair exists, we can only disagree as to whether it has a conscious power inside it, or perhaps controlling it from heaven. But you can't disprove that my prayers don't work. This analogy is pretty simple, but expand it to apply to the entire Roman world. The rituals, the sacrifices, the festivals, are what mattered; not metaphysical speculation. They kept the forces of the universe happy and moving forward.

Now, that's not to deny that alternative metaphysical theories existed. This is what philosophy was for. The Epicureans, for example, believed the gods existed, but that they didn't care about humans or what we did. Stoics, too, argued that the traditional myths were just allegories, and that fate was really the only force (or deity) in the universe. So, sometimes, the followers of these philosophies would be labelled as atheists. Not because they didn't believe in a god, but because they denied the gods as generally understood by traditional Graeco-Roman religion. Christians and Jews were sometimes called atheists, too, for the same reasons.

This is a pretty simplistic answer, but I hope it helps. A good book to check out on the topic would be Tim Whitmarsh's Battling the Gods: Atheism in the Ancient World.