Naah. You're making a pretty common mistake conflating the Romans and the Greeks here. They were an extremely different culture (think Texas vs. the UK), and, as might be expected, there were different sexual norms as well. Pederasty was not a thing in Rome, and, as such, there was no reason to ridicule Caesar for the lack of it.

What Caesar WAS ridiculed for, on the other hand, if we just stick to his sexy side, was a whole bunch of other stuff, most notably the rumour that he had slept with an old king while on a diplomatic mission. Suetonius, who's our most....colourful... source, describes it as follows:

There was no stain on his reputation for chastity except his intimacy with King Nicomedes, but that was a deep and lasting reproach, which laid him open to insults from every quarter. I say nothing of the notorious lines of Licinius Calvus ("Whate'er Bithynia had, and Caesar's paramour"). I pass over, too, the invectives of Dolabella and the elder Curio, in which Dolabella calls him "the queen's rival, the inner partner of the royal couch," and Curio, "the brothel of Nicomedes and the stew of Bithynia." I take no account of the edicts of Bibulus, in which he posted his colleague as "the queen of Bithynia," saying that "of yore he was enamoured of a king, but now of a king's estate."

At this same time, so Marcus Brutus declares, one Octavius, a man whose disordered mind made him somewhat free with his tongue, after saluting Pompey as "king" in a crowded assembly, greeted Caesar as "queen." But Gaius Memmius makes the direct charge that he acted as cup-bearer to Nicomedes with the rest of his wantons at a large dinner-party, and that among the guests were some merchants from Rome, whose names Memmius gives. Cicero, indeed, is not content with having written in sundry letters that Caesar was led by the king's attendants to the royal apartments, that he lay on a golden couch arrayed in purple, and that the virginity of this son of Venus was lost in Bithynia; but when Caesar was once addressing the senate in defence of Nysa, daughter of Nicomedes, and was enumerating his obligations to the king, Cicero cried: "No more of that, pray, for it is well known what he gave you, and what you gave him in turn." Finally, in his Gallic triumph his soldiers, among the bantering songs which are usually sung by those who followed the chariot, shouted these lines, which became a by-word:

"All the Gauls did Caesar vanquish, Nicomedes vanquished him; Lo! now Caesar rides in triumph, victor over all the Gauls, Nicomedes does not triumph, who subdued the conqueror."

Suetonius loves discussing this stuff, and if there was any rumour of any scandal, he would be on it faster than a Cajun on a crawfish boil. If Caesar had diddled little boys (or been made fun of for not doing so), he wouldn't have hesitated to report every juicy detail - as you can see by the above. If you have any more questions about Caesar, Caesar's sex life, or Roman social mores, please feel free to ask me or consult Adrian Goldsworthy's excellent biography Caesar: Life of a Colossus :)

In Roman politics, it was usual to accuse your opponent of all sorts of sexual depravities. Since it was just business/politics as usual, it's difficult to say which slurs have a basis in reality.

E.g. the elder Curio once said in a speech that Caesar was "the man for all women and the woman for all men."

Sexual innuendo was rife in ancient Rome.

Caesar was granted a triumph for his Gallic conquests. A triumph was the only occasion a general was permitted to enter Rome with his troops. It was a rather solemn occasion.

By the late republic it was usual that the soldiers sang vulgar songs about their general while marching through Rome. The triumphing general was supposed to take this in his stride.

This one was sung during Caesar's triumph (as reported by the gossip Suetonius):

Citizens, guard your wives,

For we bring you our bald fucker,

In Gaul he squandered the gold,

That you lent him this way. (meaning: on providing for his sexual appetites)

So, all in all, it is difficult to decide what is true about all these allegations. Our sources may report the rumors faithfully… but are the rumors true?

Edited to add:

But, frankly, it would be very strange in Roman society to be accused of not having slept with young boys.