How did they view people in high positions that used said workers (if there is truth to it) and pedophiles in general?
I'm going to let someone who does ancient history answer your question as it's not my strong point, but real quick: the term "pedophile" and the concept of someone being a "pedophile" (ie, attracted primarily or exclusively to children) is a modern invention. Before the modern era (in Europe/the Americas, anyway) people did not think that someone's choice of sexual object/partner defined their "sexuality" in the same way that we do today.
That's not to say that people thought sex with children was a-ok, it's simply to say that when someone had sex with an either pubescent or prepubescent young person - even if that act was criminal/taboo, as it often was - it was that particular sex act rather than the person's "sexuality" that would be at issue.
I hope that doesn't come across as pedantic - but I do think it's important not to project our modern understanding of pedophilia into the past. Our understanding of "pedophilia" depends on modern understandings of childhood itself - pedophilia, as we understand it, is arguably as much about attraction to the idea of childhood as it is an attraction to children themselves - and that really only goes back as far as the seventeenth century.
Along with what /u/American_Graffiti has said about terminology, I would like to point out that "comfort worker" is not synonymous with prostitute or sex worker. Rather comfort women are specifically women enslaved by the Japanese army during WWII to service Japanese soldiers. To a certain degree comfort woman has come to mean any prostitute working for soldiers/an army, but for the most part is has a very specific historical context and meaning.
The only example that comes to mind is Suetonius' description of Tiberius' time on Capri - he had little boys swim in pools with him and nibble him. This isn't there because of the little boys, exactly - it was part of a catalog of sex activities that symbolized the depravity Tiberius had fallen into toward the end of his life. Here is the relevant passage:
On retiring to Capri he devised a pleasance for his secret orgies: teams of wantons of both sexes, selected as experts in deviant intercourse and dubbed analists, copulated before him in triple unions to excite his flagging passions. Its bedrooms were furnished with the most salacious paintings and, as well as with an erotic library, in case a performer should need an illustration of what was required. Then in Capri's woods and groves he arranged a number of nooks of venery where boys and girls got up as Pans and nymphs solicited outside bowers and grottoes: people openly called this "the old goat's garden," punning on the island's name.
He acquired a reputation for still grosser depravities that one can hardly bear to tell or be told, let alone believe. For example, he trained little boys (whom he termed tiddlers) to crawl between his thighs when he went swimming and tease him with their licks and nibbles; and unweaned babies he would put to his organ as though to the breast, being by both nature and age rather fond of this form of satisfaction. Left a painting of Parrhasius's depicting Atalanta pleasuring Meleager with her lips on condition that if the theme displeased him he was to have a million sesterces instead, he chose to keep it and actually hung it in his bedroom. The story is also told that once at a sacrifice, attracted by the acolyte's beauty, he lost control of himself and, hardly waiting for the ceremony to end, rushed him off and debauched him and his brother, the flute-player, too; and subsequently, when they complained of the assault, he had their legs broken.
The point isn't so much about the kids, rather the amount and variety of sex. Caesar married Cornelia when she was 13 and there was no scandal about that.
Also, this is Suetonius. He's a gossip and writing a lot later, and not really afraid to make stuff up. He is the tabloid of of the Roman world.
/u/heyheymse, this is your territory. Did I screw anything up?
I think Roman boys were very nice.