You're putting the cart before the horse by asking "Why were the older, more established Near Eastern empires ... apparently unable to make similar intellectual advances?" and by calling it "perhaps the world's most significant intellectual tradition". On the contrary, the highly productive classical period is defined precisely by Athens' coexistence with the Achaemenid dominions, the Near Eastern empire par excellence. Athenian philosophers had predecessors in the Ionian schools (which just barely predate the Classical period) which drew heavily on Babylonian and Egyptian religion and mathematics. Not to be outdone, Athenian thinkers were undeniably influenced by Persian thought (evident by e.g. Zoroaster's presence as an authority figure in the works of Plato and Aristotle), although the precise extent and nature of this influence is very difficult to establish. It is possible that the Ionian philosophers drew on Iranian thinkers as well; this is most often suggested of Heraclitus, as surviving fragments suggests concepts such as the logos, and a predominance of fire and water. However, this would have occurred in the pre-Achaemenid period.

The intellectual tradition you speak of has also been heavily glossed, developed and annotated into the form in which we usually receive it today. If you go back to read source material such as Aristotle's Physics, it's a lot more wishy-washy, confusing and all over the place than most people imagine. It isn't necessarily the case that we would have recognized all these works as epochal, had they not been attached to traditions and canons so familiar to us.

Unfortunately, it is the case that most of the material the Athenians and Ionians must have interacted with has been lost or is no longer identifiable, either because it was orally transmitted or because it simply stopped being written down. What we know of Iranian tradition has to be inferred from hymns and prayers, and speculatively extrapolated backwards from Late Antique works. Few traditions appear to have survived the fall of the Achaemenid Empire; it is only from the middle Arsacid period, circa 1st century AD, that we begin to find some continuity in historical and epic narratives.

Finally, the corpus of surviving works from the Classical period is a lot smaller than most people imagine. That such weight is placed on it is largely a product of the narrative of "Western Civilization" developed in the past few centuries. It may be a useful exercise to read, for example, the hymns of the Indo-Iranian tradition (in particular the Gathas of Zoroaster), and the works of e.g. Plato, and compare and contrast these, while taking into account the different genres they belong to.

It may be worthwhile for you to reframe your thinking and question to something more like this: “Why is it common belief that the Greeks were the first or most major figures of so many intellectual, cultural, and scientific developments in the Ancient World? Why is it that we, rightly or wrongly, emphasize their agency and innovation to the exclusion of other highly developed contemporary and preceding societies?”

I gently submit that you might be falling prey to a common bias, wherein one pays more attention to known/remaining pieces of literature without mentally accounting for (1) the massive contextual negative space within that tradition of works that are no longer extant and (2) that a work written by a person of one culture gets associated with that person’s culture, often to the exclusion of others, even if that work largely draws from another tradition.

For example, you’ll likely have read works of Greek literature and heard them praised in the superlative... but did anyone in those classes or discussions or editorial notes ever explicate the cultural allusions those authors made to other traditions (I.e. not just allusions to other authors of the same tradition?

It continues to baffle me that people who are interested in ancient history are somehow aware of the existence of, for example, Linear A and Linear B as evidence that Greek is a comparatively old language.... but have never encountered the fact that those scripts died out and were eventually replaced by the Alpha-Bet (written that way for a reason) of Phoenician (Levantine) origin.

Students in my Herodotus class fell prey to this quite a lot and found themselves continually shocked whenever Herodotus referred to the Greeks getting a major part of their culture (e.g. calendar, some medicine, some astronomy, some religious practices) from Egypt.

One of my favorite fun facts to spring on people is the connection between Adonis (a quintessentially Greek religious figure if you’re a layperson and don’t know much about Greek religion in depth) and Adonai (Semitic word for “lord”) which came from Semitic polytheistic usage and syncretism... but is also (surprise!) a word used by modern Jews, and Jews since antiquity, to refer to their monotheistic divinity.

Tl;dr it’s not that the Greeks were innovating in a vacuum. You’ve probably just had them overemphasized in your education and culture, and the non-Greek antecedents to their thinking forgotten or mid-attributed to Greece. #receptionstudiesangst