Unrelated, but I wanted to point out that the Romans abandoned the traditional Greek Phalanx throughout the course of the Samnite Wars.

Generally, most ancient armies would protect their flanks with a combination of light skirmishers or cavalry. Specific tactics would depend on the time period, geography, commander, etc.

This is more about than the nature of the phalanx formation itself than specifically Roman usage, correct? Greek hoplite phalanxes were well known to have this problem--in fact, the entire concept led to a constant "drift" of the phalanx to the right as men tried to hide behind their neighbor's shield. Commanders generally put their elite troops on the right to anchor the line and prevent too much drifting, and it was generally considered a position of honor reserved for the best troops and nobility. As others have pointed out, there were generally plenty of screening forces, so outright flanking of the phalanx wasn't as common as one might expect. However, the tendency of phalanxes to drift out of place was definitely well known, and some commanders specifically sought to take advantage of this. The battle of Leuctra comes to mind, where Epaminondas and the Thebans stacked their left flank with both the elite troops and the standard, doubling the number and quality of men that would be meeting the Spartan's right. The Thebans advanced in an echelon, with the left flank preceding the weaker right. They crushed the elite Spartan right, and the remainder of the field fled.