I understand that pike-armed infantry in close order are poor targets for cavalry, but didn't these only become common in the 15th Century? Before that, how did spear-armed infantry perform against cavalry?
Part 1.
Short answer up front. Were spears a “counter” to cavalry? I’d argue no. To your second question, how did spear-armed soldiers perform against cavalry? I’d argue not much better than anyone else.
To slightly expand on the above before really diving deeper, I would agree that spears, and the longer the better, do indeed convey some advantages against cavalry. But I’d also argue that generally spears, and again usually the longer the better, are advantageous in general, and to say they specifically counter cavalry would imply some kind of specific weakness on the part of cavalry, when really I’d argue the benefits of spears apply more or less universally to any opponent.
Additionally I’d like to mention that the idea in strategy games that weapons counter certain types of soldiers in my opinion is flawed. First, it greatly overestimates the importance of the weapon when I think other factors make a much more prominent impact, especially against cavalry. It also greatly exaggerates the diversity of units, as if there was a niche and distinct unit for any situation, when really it was much more boring with the infantry, cavalry, etc typically using the same equipment. If there was any diversity, it’s likely spread out across the troops due to lack of standardization rather than some concept of dividing troops by type for certain enemies. Specialists did exist, such as engineers, pioneers, scouts, etc. but they were specialists to perform necessary non-combat functions. There aren’t any cases, to my knowledge, where different types of main body infantry coexisted separately to counter any other specific type of enemy. You might have light infantry who skirmish in front of the main body, but they don’t do so to “counter” any specific type of enemy. They’re there to perform a unique function, such as disrupt enemy formations, create gaps, impede movement, etc. They might exist in the reserve or on the flanks to be a more expeditionary unit that can fill in gaps and shore up weak points. But again, these aren’t counters. They’re unique battlefield roles that they would employ against any number of enemies and often effectively in ways that most strategy games wouldn’t expect, such as ranged troops successfully fighting back against cavalry, the traditional “counter” of ranged units. I’ll expand on this example later.
Instead, I think the best way to answer the question is to discuss what actually does allow infantry to withstand a cavalry assault. I also want to discuss why I believe the same factors that caused the dominance of pike infantry warfare in Western Europe are the same factors that led to the decline of cavalry. But I don’t think the pike is what caused their downfall, but rather shifting forces outside of the tactical sphere caused the balance to change. I believe that this seeming inverse relationship where the rise of pikes coincides with decline of knights is the root of the strategy concept of spears countering cavalry, which at first glance appears to be true, but is far too simplistic.
So if not the spear, what does counter a cavalry charge? Discipline, drill, training, morale, and oftentimes a whole lot of luck and pure chance.
Let’s first look at the Battle of Hastings as an example of the above. The English army was composed of infantry primarily equipped with shields and spears. The Norman cavalry was very similarly equipped, except, well, they were on horses. For most of the battle the English held firm, and their spears warded off charge after charge of Norman cavalry, with the Normans suffering high losses. Eventually, the Normans decided to conduct a feint, at which point the English attempted a pursuit, when the Normans wheeled about and massacred the English spearmen. What changed? How did spearmen go from soundly winning numerous engagements to suddenly being slaughtered? A cohesive infantry formation, by the very nature of a person being smaller than a horse, means that infantry can achieve vastly greater concentration of force at any point. It’s not the spear that beat the cavalry, but that no matter where knight finds himself, he is outnumbered and faced with two, three, or more infantrymen. The infantry has local superiority along the entire line by the simple virtue that you can fit more humans in any given space than you can horses. Thus, an infantry formation that has the discipline to stand firm will always have a massive advantage. When the infantry broke ranks to pursue the supposedly routing knights, they had lost all cohesion. They were spread apart, more prone to fear, and the result is now that when the cavalry engage, they find themselves the one with numerical superiority at each point.
Now for example number two, let’s look at the Battle of Rocroi where those same factors can cause pikes to fail, while an unexpected force is able to hold strong. This battle pitted the French against the Spanish, where the French had abandoned pikes for the most part and had an army with infantry composed almost solely of musketeers, while the Spanish still held strong to the tercio and fielded pikes in large numbers. Prelude aside, the Spanish cavalry struck the French left, routing their cavalry and began assaulting the French musketeers. The musketeers, many of which didn’t even have bayonets, maintained their discipline and was able to hold their position, even after Spanish pikes joined in and they found themselves being hit on two sides by both pikemen and cavalry. Later, with the Spanish pikes on the assault, they were less disciplined and cohesive than if they’d formed a static defense. The result was that the French cavalry unexpectedly flanked the Spanish pikes and utterly annihilated them. Here you see that even with an “inferior” weapon, the ability to function as a group is far more important to survivability against cavalry, while even with the perfect “counter” you’re not going to fare well if you lack cohesion. Interestingly, in the same battle you can see the reverse happen later with the Spanish pikes. After their retreat, the Spanish formed defensive pike blocks and held for a final stand. Despite being gunned down by muskets, shot by cannon at near point blank range, and being completely surrounded by cavalry, they held their own extremely well, and even after the surrender, some of the tercios never broke and were able to make an orderly, coordinated retreat.
Let’s compare to the Romans, the famous exception to the rule who used swords as a primary weapon rather than the far more ubiquitous spear. And like the tercios and the English spears above, the results are very similar. When the Romans held firm, such as during the civil war against Pompey, the legions could hold their own and drive off cavalry. When they didn’t, like at Carrhae, the Romans were decimated. In this instance the Parthians had deliberately employed psychological tactics to unnerve the Romans, degraded the commander’s judgment by killing his son, and disrupted their formations with mounted archers, resulting in easy pickings for the cavalry.
So to answer your second question about how spearmen performed against cavalry compared to other weapons, the answer is not much differently, while other factors play a much larger role.