Surely someone somewhere had to view it as inhumane, overly violent, or offensive in some way. Were there ever any recorded protests or attempt from rival groups to stop gladiator combat? Was the political/social climate of Rome even conducive to activism?
Lumped in with actors and prostitutes, gladiators were among the lowest of the low in Rome. It was something akin to the caste system in India, or Jim Crow in the American South. No one in power went out of their way to better their position.
"Social activism" in Rome tended towards "mob violence," and the upper class Romans were constantly terrified of the under classes, which included the aforementioned actors, slaves and gladiators. There were multiple large-scale slave revolts, most famously led by Spartacus--a gladiator--which only reinforced the Roman upper crust's general dislike (and abject terror) of poor people.
Couple this with Romans' broad acceptance of violence in any form, be it on the battlefield, in the arena or just walking down a street at night, and no one was out picketing for gladiators.
That said, modern depictions of gladiator fights are probably more violent than the actual events. The men fighting, though they were often slaves, represented a significant investment on their owner's part and were unlikely to die in their first fight. Or their tenth. Or ever, especially if they were popular.
I'd be particularly interested in pre-Christian examples if there are any, myself.
I recall from Anthony Everitt's book "Cicero" where I believe Cicero confided to someone his disgust at what Roman gladiator fights had become (professional performances). His chief objection was a moral one, and he believed that in the early days of the Republic, properly managed gladiator combat supported Roman ideas about bravery and honor. I believe that sentiment was captured in this statement from Cicero (paraphrased from memory): "If even a slave can fight so bravely, what does mean for the Roman citizen?" The implication being that since Romans are free and "better," they must actually be better than slaves, and thus the games served a moral lesson to the spectators. However, Cicero did not object beyond this very specific issue, and supported gladiatorial games otherwise.
The only other Roman I know that had serious objections to the games is the philosopher Seneca, who lived under Emperor Nero's reign. I believe his objection was similar to Cicero's but viewed the spectacle of the games as fundamental to its character, as opposed to a mishandling as Cicero saw it.