This is a very broad question and the answer really ranges between very effective and very ineffective depending on a whole host of factors. I’ll lay out a few examples and hopefully that will help.

First and most famously is the Macedonian phalanx. Innovated by Philip II it was used to crush the famous Theban infantry and this establish Macedonian hegemony over the Greek cities. It was later used by his son Alexander the Great to conquer the Persian empire and beyond. There were other innovations, like the companion cavalry core, that helped but the backbone of these armies and those of the following Hellenistic period were the phalanx.

The difference between the Macedonian phalanx and the hoplite phalanxes of the Greek states was the spear itself. The Macedonians used a special kind of spear called a sarissa. It was much longer than other weapons (including spears) used at the time and much heavier as well. The entire formation was designed around deep lines all carrying the 4 meter long spear, leveled down at the front and raised up in the back ranks. This had many effects. Due to the density of spears at the front no cavalry could charge them, no infantry could get close enough to properly engage them and the density of spears in the air helped deflect incoming arrows. It was not invulnerable though, the formation was often slow and unwieldy and needed highly trained men which were expensive and by the Hellenistic period hard to find. By the time clashes with the Romans occurred some had tried to mess with the length of the sarissa, which might have hurt its effectiveness depends who you ask, and the formation was simply too slow and vulnerable to being flanked. Without competent troops supporting its flanks and effective cavalry support it was destroyed by the flexible manipulation legions. It really did dominate ancient warfare for 100 years though and it was all down to a long, heavy spear.

Another significant example is the Battle of Tours/Poitiers (it depends who you ask) where Charles Martel defeated a large army of Umayyad cavalry with spearmen. The Arab caliphate had relied on its warrior caste of horsemen to sweep through the Eastern Mediterranean and North Africa. While effective infantry were used as well, Late Antique European infantry was really not capable to dealing with the Arab and Berber horsemen. The successor states in Western Europe were unstable, torn apart by internal conflict and not exactly at the peak of military technology. A relatively small Arab force had destroyed a Visigothic army and killed their king at the battle of Guadalete in 711 and rapidly conquered the Iberian peninsula. Now moving into Frankish Gaul there really seemed to be no way to stop them. Martel managed to because he built a core of trained phalanx spearmen (with normal spears not sarissas) and made them hold position in some woods which broke up the cavalry charged against him. The tight packed spear formation made horses shy from charging in and allowed the men to stab up at the men on horses much taller than them when the charge faltered. Even then the battle was only decided when a flanking force attacked the Arab camp and their army splintered trying to go back and protect their loot. The spear was effective here but it really was the training and discipline that allowed the Franks to win.

The spear is the most common weapon used throughout history because it is the easiest. A sword takes some training to use effectively but a spear you can put in the hand of any peasant to tell them to stick them with the pointy end. This is effective when the infantry is really just a holding force for a more effective unit, medieval knights and their infantry being a perfect example. You can improve the spear but it doesn’t change the dynamic much. When fighting the Mongols the Chinese initially tried to use pikemen, an effective tool against charging Chinese cavalry. The problem was that the Mongols wouldn’t charge a line of prepared pikemen, instead riding around and shooting holes in the formation before moving in and carving it to pieces. The Roman used pilum, a type of throwing spear, to great effect in the classical era. It was only useful though due to special design of a heavy metal head but soft metal that allowed it to punch through enemy shields but not allow the warped head to be removed, rendering both the shield and spear useless.

I could continue going on with examples but I think it is clear that the spear can be useful but it is dependent on a bunch of other factors, technology, what your enemy is doing, battlefield circumstances, economic and social factors and, probably lost importantly, training and discipline. As for it’s use in single combat I have less expertise but my time studying medieval nobility had always proved that in single combat a sword is always a more versatile weapon.

If you want to check out some evidence yourself I recommend literally any book on Alexander, they all have to talk about the phalanx it is key. J. R. Hamilton’s Alexander might be an easy place to start. If you want to learn about the battles I talked about and don’t like reading YouTube is full of history channels that make detailed and well researched episodes about these battles. BazBattles covers Guadalete and Tours quite well and Kings and Generals have a great series on the Hellenistic period.