"Replacement" is not an accurate term to use in the evolution of warfare, especially ancient warfare. "Replacement" assumes that everyone is operating from the same playbook, and that changing from a phalangite-based martial system to a sword and shield martial system was a conscious or deliberate choice, like upgrading a tech tree in Civilization, or something. The reality is much different. I'm going to try to unpack it a little bit.

1: Organized violence is social, and militaries are an expression of a social structure with economic particulars.

If we take the example of "Greek Hoplite" in its most generic possible sense, we get a pretty common image: a man with a bronze helmet with a crest, a round, bowl-like shield, a cuirass of metal or cloth, greaves, sandals, a spear. We all know the image from pop culture and are likely conversant with some of the details, like superficial knowledge that a man dressed like this fights in a phalanx. We might even have an opinion on how they fought, and specifically how they used their spears, or didn't use their spears.

That's all well and good, but remember that there is an entire society behind that phalangite, there are details of that man's upbringing, ideas that he has about what makes a man a man, about what makes that man a citizen, perhaps about what makes that citizen a warrior.

Now picture another man. He's similar in all of the broad details: he too wears a helmet, has armor for his shins and chest and carries a shield. But this shield is shaped differently; it's rectangular rather than circular, and the ends along its long axis curve inward giving it some complexity. And instead of a spear, he carries a sword.

He too has assumptions about manhood and citizenship, has an idea about what makes a warrior. Some of them overlap with our generic Greek warrior. Some of them are radically different.

Behind both men is a society of craftsmen, makers, builders, traders and laborers creating and distributing all of the tools that he takes with him to war. That society makes up the majority of decisions with regard to how that man fights. Spear, sword, aspis or scutum: neither of these men can go to a market and decide that he will mix and match or agonizes about the damage-type he can inflict with a spear versus a sword: he takes with him what his society deems practicable for battlefield use.

We can do this exercise with any warrior from history or even any warrior from our imagination: even the most mundane details of a warrior's kit reflects decisions, traditions, and economic advantages and disadvantages that are shared on a societal level.

2: Military developments aren't linear, and changes don't necessarily imply superiority

Modern folks have a tendency to assume that anything newer is better. And while that's certainly the case with, like, iPhones, that does not carry over into military history or the history of technology. One axiom that we can be sure of, though, is that changes reflect challenges on a social or economic level, and that their expression in new equipment or tactics or what-have-you are a visible attempt at problem-solving.

Most of the time, the problems will be overcome with solutions that offer as little disruption as possible. Temporary alliances against an invader are one example, since an alliance with rivals who nevertheless share a culture, language, and ethos is socially non-disruptive. Now, on a military level, there are more available soldiers to fight the invader, but socially and economically very few changes have been made.

In the case of Rome and Greece, while both sides made small changes to their tactics and equipment over several hundred years, when they met on the battlefield, neither was making rapid changes. That Romans consistently won battles is a question more about tactics, strategy, and the application of their social and economic advantages (which affect recruitment and retention of soldiers, motivation to fight, the availability of supply and equipment, and how a society shares the cost of maintaining an army). On an individual basis, if this were a question posed to something like Deadliest Warrior, skill or advantage in their kit doesn't matter a whole lot, because wars are fought between societies and cultures, not individual warriors.

To tl;dr this point: just because a single society makes changes to their approach to combat doesn't mean that those changes reflect some objective improvement, it just means that those changes reflect an effort to solve problems posed by some external or internal force.

If you have a question more specific about a particular example of changing weapons and tactics, please ask here. I will answer if it falls into my area of knowledge, or can refer you to other answers in the FAQ.

There have been quite a few answers on and around this topic- here's a couple you might find interesting!

First, there's this answer by /u/iphikrates, which delves a bit into the idea of a formation going "obsolete" to begin with, and shows how the phalanx was very context-specific.

This post, with discussion from both /u/iphikrates and /u/iguana_on_a_stick contains a lot of discussion about the phalanx itself, and what separates a phalanx from any other spear-and-shield formation (which was obviously still in use well after the end of the classical period).

Just to add to my earlier answer on phalanx obsolescence that was shared by u/butareyoueatindoe:

u/PartyMoses has already made clear that ancient weapon technology didn't really work in the way that modern weapon technology does, with huge investment in R&D to produce arms that render their predecessors objectively obsolete. Ancient military equipment was to a large extent prescribed by traditions and socio-economic factors. Different weapons and tactical systems could not necessarily be ranked in order of superiority, and did not replace each other even after one had triumphed over the other. Instead, many different weapon types coexisted, and neither victory nor defeat in battle were often seen in terms of which side had better armament.

Modern analysis of the Hellenistic period, however, is faced with a difficult problem. The superpowers of the era, fielding the professional Macedonian-style pike phalanx that Alexander the Great had used to overrun the entire Persian Empire, proved consistently unable to defeat the armies of an upstart Italian city-state that could boast no such specialist infantry force. How can we explain the military failures of the Hellenistic kings? How did the Romans not only withstand their attack, but eventually managed to conquer every single one of the territorial empires of their day?

You can see how this question would tempt some people to look for innate factors. It doesn't seem satisfactory to say that the Romans prevailed in each of their many battles due to contingent factors, local superiority and luck. Humans crave narrative; we find it much easier to accept an account that gives us a neat set of structural causes and effects.

Thankfully, the most trusted authority of the period, the Greek author Polybios, actually offers such a structural cause. In the course of his narrative of the Second Macedonian War, this experienced commander presents a detailed comparison of the advantages and disadvantages of Macedonian and Roman infantry tactics (18.28-32). Almost everything you'll hear in pop culture and pop history about the Macedonian phalanx and the Roman response to it comes from this Polybian comparison. All that stuff about pike formations being inflexible, cumbersome, and vulnerable to disruption, and about Roman maniples being more flexible and manoeuvreable - it all comes from this one passage. In it, Polybios gives us the widely repeated point you're raising here: that a Roman legion made up of swordsmen is innately superior to a Macedonian phalanx made up of pikemen.

We must remember, though, that Polybios is trying to explain a specific event: the defeat of Philip V at Kynoskephalai. Any reader of his account of the battle itself would credit the victory to quick thinking of individual commanders in a chaotic engagement, which meant that the battle really could have gone either way. But for Polybios, there had to be a deeper cause, and he found it in his comparison of the performance of the two infantry types in pitched battle. In doing so, he elided from his account the role of light infantry, non-phalangite heavy infantry, and horsemen. This is the pattern that many modern retellings of the encounter still follow. They will say the Roman victory was due to the superior flexibility of the maniple, which allowed it to run rings around the cumbersome phalanx. They are simply following Polybios.

Now, it is of course fine to do so; he is, after all, one of our more reliable sources for the period. But we can also take a wider view. As I argued in my other post, the pike phalanx was a specialist tactical formation intended to fill a specific role in pitched battle. It was not intended to fight alone. The same can be said of the Roman maniple; Roman infantry was typically supported by light infantry and cavalry, and raised significant numbers of each for its campaigns. It follows that the encounter between legion and phalanx was not the whole story, and must not by itself be seen as a full explanation for the Roman triumph at Kynoskephalai or elsewhere. If we zoom out even more, we can see that pitched battle is hardly the full story of ancient warfare; sieges, skirmishes, ambushes and naval warfare were all prominent features, to say nothing of diplomacy and careful alliance building. The Romans' meteoric rise cannot possibly be the result of infantry weapon technology alone. Polybios himself would surely not have argued this either. And it follows that we cannot simply say that Roman infantry tactics proved superior and superseded the pike phalanx. The actual story is far more complex and involves far greater factors than simply what particular tools the warriors of Rome held in their hands.

Finally, it's worth noting that the development you're suggesting here never actually took place. Sword and shield did not "replace" the phalanx - at least in the sense that the peoples who inhabited the Hellenistic kingdoms never abandoned the pike in favour of Roman infantry equipment. There were some poorly attested attempts to copy Roman infantry as a supplement to the phalanx - again proving that the pike phalanx was not intended to shoulder the burden of battle all by itself - but more generally the peoples that had formerly resisted Rome simply ceased to exist as independent political entities. The Greeks did not abandon their spears; they simply no longer had a military role in which to use them. The Macedonians similarly didn't give up their pikes to become legionaries; rather, they became part of the Roman Empire. Pikemen were replaced only in the sense that no states remained that would field them. That's hardly the same as saying that the technology was phased out once a superior alternative became available.