As a youth, Cicero went to Athens to study philosophy. I've heard about other Romans doing the same. What sort of educational infrastructure was present in Athens that made it a destination of learning long after Rome eclipsed Greece as the center of political power in the Mediterranean?

This question hasn't been answered yet because it requires a lot of lead-up to address properly. It is really not so much about Athens as a physical place, but rather Roman education and how/what they extracted from Greek educational practice. The practice of sending young elite Romans to Greece is just one step in the total Roman educational experience, and one of the last steps. It will be useful to briefly list all the typical steps. All of this is for families of means, either of the Senatorial class or among the equites (the narrow Roman middle class or bourgeois).

Basic precepts of Latin grammar and reading. This happens in the home, and had originally been the responsibility of the parents (especially the mother). By the 1st century BCE, responsibility for this initial stage was often in the hands of a slave, a paedagogus, acquired specifically for this purpose. This slave was (obviously) educated and skilled in tutoring. He might be Greek, but not necessarily. Literacy rates for non-elite, and how/where literacy was attained, is a contentious topic, but note that there were no state-run "schools" in Rome for education at this level. It was in the home, or else acquired in service as an apprentice to a trade. Many lower-class boys probably learned mathematics by this route, in training to become bakers or surveyors or whatever else.
The next stage, still in the home, introduced Greek, and already we are in a stage almost exclusively reserved for the elite. It was a basic requirement for an educated Roman of means, and it focused on mastery of what the Romans considered Greek "classics"-- Homer's epics, for instance, along with Athenian tragedy and the histories of Herodotus and Thucydides. The teacher of this stage was either the same paedagogus as earlier, or else another slave, this time Greek-speaking (if not before). The divisions between the teacher here, the one earlier, and the one below are often very blurred. The richest Roman families could afford the very best tutors and probably used the same person for all levels.
After mastery of the Greek language (reading and speaking) came the formalized focus on rhetoric. This is often called the "Hellenistic" education, though it is not precisely correct, because the typical Roman flavor of this education did not espouse the full range of study topics found in the Greek gymnasium. In fact, the Romans were mostly interested in rhetoric (the art of speaking well), philosophy, and some aspects of mathematics, and by the 1st century BCE we find that most of the other disciplines typical in a "Greek" education had been dropped: some literature, art, music, dance, gymnastics, et cetera. Cicero, in his de re publica, presents the Roman ideal as essentially pragmatic, and the Greek as essentially theoretical. He characterizes Roman adoption of Greek education as utilitarian, only taking the useful bits and throwing out the rest. Unfortunately, the text is fragmented and we never get to hear the character Scipio's argument for why the Roman system, with its selective curriculum, was superior. This was the basic training to become an elite Roman citizen, which assumed by default the pursuit of political office. The teacher was a grammaticus or rhetor, always Greek, and very expensive (and desirable). Prominent families jealously guarded their rhetor from poaching by other, less fortunate suitors. The curriculum itself involved memorization of set stories (often myths) and their retelling at various levels of sophistication; recitals of arguments or precepts; and the very basics of what we would call debate today: organizing an argument around a given prompt and delivering it with a mind to persuasion, intonation, and effective content (for instance: "Was it right for Agamemnon to take the concubine of Achilles? Why or why not?").
Finally we arrive at OP's question. At this stage, there were two interchangeable paths available: remain in Rome to be the "observer" of a prominent Roman statesman; or go to Greece to continue training in rhetoric and philosophy by observation of other rhetors/philosophers/orators. The two were not mutually exclusive (Cicero did both), and both served a similar purpose: practical experience via observation. Athens was the center of Greece and was still the center of Greek intellectual life, though the heyday of the lycaeum at Athens was long past by the 1st century BCE. There was nothing special about the infrastructure of Athens for the purposes of the Romans, however. It was the people there that mattered, not the city. There was no Athenian Oxford or Harvard or Stanford. In fact, many prominent Greek intellectuals in this period were not from Athens, but rather Asia Minor, and resided in the great cities of the period like Antioch, Ephesus, or Alexandria. Athens was, of course, the nostalgic destination, and had been the absolute center of learning in the Classical period. It had a certain gravity as far as intellectual pursuits, even though its political influence was virtually nil by this time. One wonders if Cicero chose to forget that, only a generation before him, the Roman general Cornelius Sulla had chopped down the trees in the lycaeum to build siege works, and eventually had razed the city and burned the Acropolis, destroying or scattering countless objects of art accumulated over nearly ten centuries.
Formalized attachment to an elder statesman was typical in the Roman system, and it served to "show the ropes" of the machinery of Roman government and forum life to the pupil. Cicero was fortunate enough to be attached to the venerable Quintus Mucius Scaevola, and his fellow pupils were Sulpicius Rufus and Titus Pomponius. The former would go on to become the greatest legal mind of his generation and Cicero's lifelong friend; the latter, nicknamed "Atticus," would become Cicero's best friend and the recipient of hundreds, if not thousands, of Cicero's letters. The point being: this was an important aspect of Roman education not just for the learning value, but also the socialization. This must have also applied to the trips to Greece as well: they were summer camps, an opportunity for the young man to get out "on his own" (on family's money) and brush shoulders with other young elite Romans.
I won't spend a lot of time on this step, but it was essential in the Roman education system: military training. Even the very reluctant Cicero had to do it. For elite Romans, it involved attachment to a general in the field as either tribunus militum (tribune of the soldiers, an earlier practice) or legatus (aide/lieutenant, a later practice prominent in Caesar's era). Military service was a prerequisite to higher office, no matter if you wanted to fight or not (and Cicero certainly never did).

In summary: there was no educational infrastructure in Athens which lured Roman pupils there. Rather, the Roman students, in their pursuit of Greek rhetorical training, went to Greece to observe and study under Greek thinkers in these areas. These men had been educated in the typical Hellenistic model and had become masters in their respective fields (mathematics, rhetoric, art, literature, etc). Many of these Greek thinkers were brought to Rome to serve as the private tutors of the ultra-elite, but the best and most famous among them remained free individuals. The trips to Greece served to help socialize the young Roman pupils, to provide them "networking" opportunities, and to give them fresh perspectives on the subjects of philosophy and rhetoric (most prominently).

Some sources for further reading:

Bonner, Stanley F. (1977) Education in Ancient Rome: From the Elder Cato to the Younger Pliny. London: Methuen.

Christes, Johannes (1979) Sklaven und Freigelassene als Grammatiker und Philologen im antiken Rom. Wiesbaden: Steiner. (German, sorry)

Morgan, Teresa (1998) Literate Education in the Hellenistic and Roman Worlds. Cambridge.

Rawson, Elizabeth (1985) Intellectual Life in the Late Roman Republic. London: Duckworth.