Short answer:

Yes, but not for the reasons you assume.

Discussion:

The subject was addressed extensively in a recent [2015] book, "Battling the Gods: Atheism in the Ancient World" by Tim Whitmarsh, a classics professor at Cambridge University. This is a skillful effort, though one should take note as reviewers did of Whitmarsh's habit of eliding skepticism and agnosticism with atheism. He's turned up a remarkable number of unambiguous sources, and quite a few compelling interpretations. While it's obvious that we should avoid anachronistic ideas of belief, it is perhaps less self evident but also necessary to steer clear of anachronism in unbelief.

The short answer is yes, though the connection to progress in science isn't the necessary nor sufficient predicate. The ancients did have "progress in science" but more than a thousand years later we can find intensely religious men like Isaac Newton, who made substantial progress in science without becoming atheists-- though one can argue that Newton's science substantially altered his religion.

Whitmarsh isn't looking so much to scientific progress in the ancient world for skepticism about the gods, but rather to a philosophical mode of inquiry. In considering the story of Salmoneus -- a man who, so the story goes, declared himself to be Zeus and demanded sacrifices be made to him-- Whitmarsh asks "If gods can be fashioned by mortal imitation, how real can they be?”

That's a subtle bit of implication, but Whitmarsh points to more clear cut cases, the Sophist Protagoras who quite specifically said "I cannot know if the gods exist"; Prodicus had even more stark views. The Epicureans held parallel views arrived at somewhat differently.

There were atheists and agnostics in the ancient world, but your assumptions of just how they arrived at their views needs to be reconsidered, "[w]ith advancement in philosophy and science in general" -- is too broad and "now-ist" an imputation. How the Greeks in particular got to their skepticism requires an understanding of Skepticism, the philosophical tradition.

It's important that the Greeks and Romans didn't necessarily understand their "gods" as we understand gods in revealed religions with scriptures and creeds. When Caesar Augustus was deified-- did people subsequently think of him as a "god" in the way that believers mean today? Some did - and one can think of contemporary traditions like the canonization of deceased Popes that seems roughly analogous-- but many plainly didn't see him as any more than a dead Emperor.

So it's not useful to project 21st century sentiments like "“This is stupid. Why do we even believe in men on Olympus?” 2500 years into the past. That's a long way off in history, and you'd do better to ask "what and how did they think" rather than "did they think the way someone in 2020 might think".

Did the Greeks ever really think that the Gods were physically on Olympus? Some did . . . but plainly some didn't. We can point to modern analogies in some contemporary religious traditions, where there's not much interest in insistence on the literal truth of every word of scripture. People can tell stories which have meaning, without believing in their literal truth; men have been good at stories for a very long time. The Greeks in particular told stories, stories featuring gods. We don't do that-- we have scriptures, but our religious traditions are ambivalent about "other stories about God"; extrascriptural sources are generally not accepted outside of certain limits (accounts of miracles, for example).

On that note it's also important that the Greeks and Romans had very little by way of scripture or creed. Religion might be thought of somewhat similar to Shinto, various traditions and stories, shrines of ritual and sometimes political importance, but lacking elements to parallel what we see in, say, the Abrahamic faiths. To have doubts about a religion with a scriptural and enthusiastically taught creed -- that's quite different as an intellectual position than to have doubts about beliefs that are promulgated with much less philosophical vigor in the first place.

I'll answer with an eye on Greek philosophy.

Firstly, it depends what you mean by 'atheism'. Your question asks whether people in the ancient world said 'This is stupid. Why do we even believe in men on Olympus?' Of course, no philosopher believed that the gods were men on Olympus. If that is what you mean by 'atheism', then every Greek philosophy was an atheist.

Each Greek philosopher, to varying degrees, rejected the popular theology. But no Greek philosopher went so far as to say that not a single god exists.

Xenophanes, an early Greek philosopher, opposed popular theology on two grounds. The first is that it attributes to the gods things that are unbecoming of them:

Homer and Hesiod have attributed to the gods all sorts of things that are matters of reproach and censure among men: theft, adultery, and mutual deception. (B11)

…as they sang of numerous illicit divine deeds: theft, adultery, and mutual deceit. (B12)

The second is that it is a very anthropomorphic conception of the gods, which Xenophanes thinks is implausible:

But mortals suppose that gods are born, wear their own clothers and have a voice and body. (B14)

Ethiopians say that their gods are snub-nosed and black; Thracians that theirs are are blue-eyed and red-haired. (B16)

That being said, Xenophanes was not an atheist and still believed that there was a god, just one who differs profoundly from human beings.

Plato picks up on the first of Xenophanes' criticisms and argues in Republic books II and III that the poets are just plain wrong when it comes to the gods.

Plato, in his own myths in the Phaedrus and Timaeus, has no problem using the names of the classic gods to refer to agents, but it seems pretty clear that he does not believe in the Pantheon as such. Moreover, he thinks that the actually-existing God, whom he calls the Demiurge (coming from the Greek demiourgos, meaning 'craftsman') is not omnipotent and is not able to make the world fully good. Plus, he acts on pre-existing material stuff, not creating the world out of nothing. In the Philebus, he does not shy away from referring to this being as Zeus. But again, obviously not classical Zeus.

Plato comes the closest of any major ancient philosopher to believing in the Greek Pantheon, in virtue of his use of the names of figures from the Pantheon. This explains why there was a phase in late antiquity of mystical Platonists who practiced theurgy and sacrificed animals to the gods. But this is not monolithic, and Platonism also went through a heavily skeptical phase much earlier.

Aristotle believes in God, but it is not a god who is active. He is engaged continuously in a process of self-reflecting (specifically, thinking about himself). Due to this, he is not omniscient and is not aware of our existence. He develops the view in the twelfth book of the Metaphysics and eighth book of the Physics that God is not a creator: the world is eternal.

Generally speaking, divinity, at least among Greek philosophers (if not laypeople, too), was not about perfection, but immortality.

The Stoics believe that there is a corporeal god whose being pervades the whole cosmos. They are happy to refer to this being as Zeus, but it clearly isn't the classical Zeus.

The Epicureans are the closest there are to atheists. They believe the gods exist, but they do not care about human beings or even take notice of us. They live perfectly self-sufficient lives, and do not need to notice us for any reason. Many Epicurean fragments make it seem like the gods are simply the heavenly bodies, in which case Epicureans believed that the gods were fiery balls of stuff. (This does not rule out their being intelligent and conscious: the Epicurean worldview was a materialistic one, so even we conscious and intelligent human beings are ultimately made of material stuff, namely, atoms.)

Sources and recommended reading:

Lloyd Gerson, God and Greek Philosophy: Studies in the Early History of Natural Theology (New York: Routledge, 1990)