It is my understanding that prior to the phalanx most battle was sort of a collision of two sides having many individual battles inside of the large battle and it was really the phalanx that would create a more modern interpretation of infantry tactics. So how was Rome able to overcome this seemingly perfect tactical creation?
First important to note that when we speak about the *Greek* phalanx we typically speak of the hoplite phalanx of the Classical era, which involved hoplites arrayed in a compact formation about 8 men deep (although the depth varied).
However, what the Romans defeated was the *Macedonian phalanx*, which essentially took the Greek phalanx and put it on steroids: the depth was doubled to 16 men deep (and could go as deep as 32!) ; the 8-12 foot hoplite spears were replaced with 16-20 foot pikes. The Macedonian pike phalanx not only defeated the Greek hoplite phalanxes as Chaironaea, but had played a critical role in the pitched battles that overthrew the Achaemenid dynasty.
The Romans only fought six major pitched battles against Macedonian-style phalanxes (I am excluding a seventh, Thermopylae in 191, although it was a crushing Roman victory after the Romans attacked the Seleucid phalanx from the rear of its blocking position). Of these six the Romans decisively won three: Cynoscephalae (197), Magnesia (190) and Pydna (168). Two were decisive losses (Heraclea (280) and Asculum (279), both to Pyrrhus. Finally, one battle with Pyrrhus was a tactical draw (Beneventum in 275), albeit a strategic victory for Rome.
So from our small sample set, the legion v. phalanx score is 3-2-1, which isn't bad although does not seem like anything to write home about. But the Roman wins were truly crushing: Cynsocephalae ended the Second Macedonian War on Rome's terms; Magnesia ended the Syrian War with Antiochus III, who made enormous territorial concessions and paid Rome a huge indemnity; Pydna ended the Antigonid monarchy altogether.
Of course, the devil is in the details. For the loss at Heraclea, the legions fought Pyrrhus' phalanx to a draw, but the battle was lost when Pyrrhus' cavalry and elephants routed the Roman horse and thus flanked their line.
At Asculum, Pyrrhus also placed maniples of his Italian allies in between the pike blocks of his phalanx, thus negating one of the traditional weakness of the phalanx, namely its habit of losing its cohesion and opening gaps that could be exploited by the enemy. But those gaps weren't so vulnerable if you've had the foresight to fill them with Rome-hating Samnites before-hand!
Magnesia is a strange case: the Seleucid army included a 16,000 strong phalanx arrayed 32 men deep, as well as 10,000 Silver Shields, who fought as somewhat more nimble phalangites. Notably, Antiochus, like Pyrrhus, had the good sense to fill the space between pike divisions with light infantry and elephants. But the main Seleucid attack, which badly handled the Roman left, was almost entirely cavalry, including some 3000 heavily armored cataphracts, where both horse and rider wore armor. Meanwhile, the Romans won in part because of their own cavalry charge on their right, led by their ally Eumenes of Pergamon. Its not entirely clear that the legions and phalanx had a direct collision in this battle as organized formations, although once the enveloped phalanx collapsed (perhaps due to an elephant panic), there was quite likely a lot of fighting between legionaries and phalangites as the Romans stormed the Seleucid camp. So a big victory by Rome over an enemy with a big phalanx, but the legion v. phalanx aspect is not decisive.
This takes us to two battles where the legion clearly did crush the phalanx: Cynoscephalae and Pydna. At Cynoscephalae, Philip V was leading his phalanx over a ridge; both sides were trying to deliberately escalate a morning skirmish into a pitched battle, but due to fog and fog-of-war, neither side was entirely sure where the other's main force was. Suddenly seeing the Romans formed up below, Philip ordered a hasty charge with half of his phalanx, since the other half was still forming up. This charged down the hill like sonic-the-hedgehog, and pushed the Roman left-wing at the bottom back. However, the Roman right attacked forward, and started carving up the unformed phalangites who were still falling into formation; one disadvantage of a 20 foot pike is it only works if you are in tidy formation with other pikemen. So two battles: one where the phalanx is beating the legion, and one where the legion is beating the phalanx. One final advantage of the legion: it has a lot of officers who can make independent decisions. One of these, a military tribune (elected by the Roman people), fighting with the Roman right (wining), on his own made the decision to transfer twenty maniples to attack the rear of the phalanx that was attacking the Roman left (where the Romans were losing). So here the modular aspects of the legion, and its empowered officers, was a real advantage, and each Roman legion had not only 6 military tribunes, but 60 centurions.
Finally, Pydna is the classic case of the unwieldy phalanx that simply falls apart as it attacks. This battle famously started suddenly, as an unintended skirmish over an escaped horse escalated into a full battle as both commanders committed their heavy infantry. This meant again that the ability to hastily form up and advance over rough ground was key. Early in the battle, a crack brigade of phalangites roughly handled several cohorts of Rome's allies. But once the legions were led against the main wings of the phalanx (the Bronze Shields and White Shields) they quickly carved them up. Hasty formation and rough terrain left many gaps in the pike formation that could be infiltrated either by Roman maniples, or even small groups of legionaries, and thus both formations, over 20,000 pikemen, were savaged in roughly twenty minutes; the entire battle took less than an hour. (I have written, with Paul Johstono, a full article on Pydna which is available open source).
One thing that is clear, is once you subtract the 20 foot pike, which can only be used effectively if you have 256 friends standing near you to form a 16x16 pike block, the legionaries were much more heavily equipped. The Roman gladius hispaniensis is about six inches longer than Macedonian xiphos or machaira-style swords (gladii have 65 cm blades, compared to 45-55 cm blades for Hellenistic swords). The Roman scutum (roughly 120 by 75 cm) offered twice the physical coverage of smaller Macedonian shields (60-75 cm diameter). And the Pydna monument is our first evidence for Romans in mail armor, which is very protective; Bret Devereaux has a forthcoming article suggesting that mail armor not only gave Roman legionaries a significant edge against phalangites (who wore linen armor), but that the timing of the introduction of mail, probably in the 3rd century BC, may explain why the Romans do poorly against Pyrrhus but then mop the floor with Philip, Antiochus and Perseus. Mail simply made Roman infantrymen much more capable of close combat, the type of fighting that carved phalanxes apart from the inside out.
Ultimately, there were not many engagements between the legion and phalanx, but the Roman win enough of these, and more importantly, win big, and between leadership, tactics and equipment, it is not entirely surprising that they did.