In effect mounted archers often did defeat the legions, but for a large part of their history, that was simply not what they were fighting against. In the west the majority of their combatants were large infantry armies like theirs. They did have some cavalry, but nothing compared to what a traditional steppe army would look like, with large groups of horse archers harassing the large infantry blocks of the Romans. Also what was fielded in the west was mostly melee cavalry and used for shock or reconniassence.

A notable exception in the west was the Numidian cavalry, fielded by the Carthaginians in the Punic Wars by Hannibal, and later Scipio Africanus, which handily defeated the Roman cavalry auxillaries. But they were never in large enough numbers to form something like the Mongol horde.