I've always wondered how and why the Roman maniple system was considered superior to a phalanx. How could a soldier with a heavy shield and gladius penetrate a wall of spears? I understand that the phalanx was hard to manoeuvre and terribly vulnerable at the flanks, but assuming the flanks were secure with cavalry or something, how would a typical legionnaire be able to do anything apart from throwing a pilum or two?
Lots of reasons that actually have very little to do with two (actually, three) men exchanging sword and spear attacks. The first thing to know is that the phalanx was extremely unwieldy. If they're not marching across flat, unbroken ground then they don't stand much chance of holding off a charge. The maniple is a rather flexible arrangement. If, for example, a manipular formation takes on a phalanx formation, reserve troops far more effective in the maniple as they are less crowded and can easily step through the formation to take the place of any soldier. Furthermore, each soldier in a phalanx must stay in place for the formation to be effective, not an easy task for a group of soldiers carrying heavy shields and spears twice their height or more. The maniple soldiers, on the other hand, could easily fight in smaller units or even as individuals of necessary.
The main advantage though was being able to effectively use a manipular formation on almost all types of terrain. The Romans could simply not engage with a phalanx on a flat ground and instead choose more favorable terrain. The phalanx had to meet them, or the Romans would just happily march right around them and occupy their cities.
As to your protected flanks with cavalry, a single unit of cavalry can only do so much. If all the cavalry is tied up by, say, strong Roman cavalry nothing is stopping a part of the flexible maniple from walking right around a phalanx that is already engaged. If the phalanx cavalry chooses to ignore the Roman Equites to fend off flanking infantry, then the Roman cavalry just charges right into the back of the phalanx.
So, the Romans recognized the awesome power of a phalanx from the front on unbroken ground. As a response they either flanked around, or much preferred to just avoid battle in unfavorable conditions.
Sources: The Histories of Polybius Volume 2, Polybius.
Edit: I seem to have only answered how the maniple could commonly defeat the phalanx, but not why the phalanx fell out of favor. Simply, the Romans conquered most of the phalanx using (Hellenistic) world and preferred their own systems. The phalanx could still be found in the East, and eventually made a reappearance in Europe, but I cannot speak to these matters with any knowledge.
Define "phalanx." Is a phalanx specifically a Greek phenomenon, or is it any formation of infantry that is kept very close together for the purposes of mutual protection and concentration of combat power?
If you accept the latter definition, the phalanx didn't really fall out of favor until the 19th century.