Yes, it did happen occasionally. Such was the attitude then, that high-class women could pretty much do as she pleased, married or not - and if a gladiator pleased such a woman, he had a much better chance of being elevated to honored - and protected - status as a Roman citizen. And he wouldn't have to risk his life in the arena again. Like I said, though - it happened only occasionally.

You have to remember, attitudes were very different. Marriage was more of a business partnership than a romantic union. Especially among the elite women, you were looked upon as "strange" if you didn't take a lover now and then. Men, when they got together in the Roman bath houses, would brag about how many times a night their wives got it on with other men. It was not at all an embarrassment to be known as a cuckold.

Sources: Prostitution, Sexuality, and the Law in Ancient Rome by Thomas A. J. McGinn