How exactly did ancient warfare work on the battlefield?

It depends a bit of course on the place, the sides involved, the size of the armies and the time.

Different sides used different tactics and styles of warfare developed out of their wealth, social system, access to metals, horses and craftsmen and the terrain.

The general rule is that most casualties happened once one side had started to run. Grossman (his work has been debated though) claims that it is psychologically easier for himans to strike another human in the back, both because of dormant hunting instincts, but also because it is easier to dehumanise and kill someone whose face you cannot see.

The ancient Greeks used the hoplite phalanx, a tight formation of free men that bought their own equipment. This was the middle class, with well-made weapons and armour paid for individually, but fighting in discplined, close formations with spear and large shields. The men used their large shields to protect themselves and the man to the left of them. Thus there was a tendency to pull right, both because you wanted to be as close to the man on your right to be protected by his shield as well as your own, but also because most people are right-handed and wield their weapon from the right. It was not unheard of for two hoplite formations to completely reverse side by both sides' right flank beating back the opposing left flank. Hoplite warfare could become a inverted tug-of-war where both sides attempted to break the others formation, break their line and roll up on both sides. If a formation broke, the side that broke would most likely soon start runnig, and had then lost.

The Greek hoplite armies made use of missile troops to harass the enemy formation, recoinnasance, flanking and skirmishing. Javelin throwers (peltasts) and archers and slingers were common. Cavalry were mostly light and only used to harass the enemy troops, with the Thessalians as the sole exception to this rule.

The lower classes served as skirmishers and lightly armoured and armed troops, the middle class as heavily armoured and armed infantry and the upper class as officers. The Spartans and the Thebans were the masters of this kind of warfare. The Hoplite started to evolve to becoming the Thurepos in the style of Illyrian medium infantry, more lightly armoured, with an oval shield and carrying javelins as well after the long wars over Hegemony over Greece.

The Macedonians made a change in this by having a strong state that could afford to equip and train the sarissa (pike) phalanx. A Macedonian pikeman had lighter armour (often linen) and less protection than a hoplite. He also had a much smaller shield strapped to his arm, since he needed both hands to hold his pike. With a high degree of discipline and training the pike phalanx was unbeatable from the front and when in formation (terrain and other factors could make the formation less than rigid). It had to be flanked or attacked from behind, or its formation broken up. The Macedonians also introduced heavy cavalry even better than the Thessalians to the Greek world, cavalry that could charge the flank or rear of an enemy infantry formation, in the form of the Companions. The Macedonians probably introduced what was the first complete combined arms army where the state could get all parts of what it needed at any given time. Macedonian slingers were renowned for their skill and had a longer range than bows of the time. The Macedonians were the first to be able to field large armies of poor men equipped and trained to take on any infantry in the world, giving them a distinct adantage. They also fielded a smale contignent of elite medium infantry called the hypaspists, who were equipped and fought similarly to the Roman legionaries.

The Egyptians were under Persian rule, were conquered by the Macedonians and the Ptolemaic dynasty, founded by one of Alexander's successors, made use of the Macedonian style of warfare. Initially with Greek and Macedonian mercenaries, but eventually with native troops trained in the same style Cleruch cavalry, men who served as heavy cavalry in the style of the Companions and Thessalian cavalry.

The Carthaganians used native Tunisian and Sicilian Greek infantry in the Greek hoplite style originally. However, since the number of citizens in Carthage dropped, they started making use mostly of mercenaries from North Africa, Sicily, Iberia and Gaul (see below for some mercenaries sought after during the era).

The Iberians, or Celto-Iberians in Spain and Portugal made use of medium infantry with short swords, javelins and shields which, when serving as mercenaries with the Carthaganians, helped the inspire the Romans to develop the gladius and to adapt their Hastati to their Marian legion. The Celto-Iberians also made use of heavily armoured cataphract cavalry, although in very small numbers.

The Gauls, or celtish people in Gaul made use of spearmen, good javelin throwers, noble infantry and heavy cavalry as well as archers and other skirmishers. Naked fanatics are mentioned in one source as men with long swords, large shields and no clothes (however, only one source mentions them, and only in one battle, in which they were severely defeated).

The Germans - little is actually very well known, but the spear seems to have been the focus of their warfare, both on foot and on horse, both thrown and stabbed. Germanic heavy cavalry were popoular mercenaries both for the Romans and the various Gaul tribes fighting them or each other before the Roman takeover.

The Persians before the Macedonian conquest made extensive use of archers and other missile troops, horse archers, heavy and light cavalry and masses of conscripted light spear infantry with wicker shields. This army came up short in Greek terrain against the hoplite phalanx at Marathon and Plataiai and against the combined arms army of the Macedonians. After the Macedonian conquest, the Persian Empire was divided among his hiers (called the Daidochi) who mixed Macedonian style warfare with the usage of more missile troops. The pike phalanx of these states became heavier and heavier to resist missiles, culminating in the Selucid silver shield phalanx, men who were like medieval knights and Egyptian cleruch cavalry awarded a parcel of land to finance themselves and their expensive equipment.

The Parthians, a horse people who eventually rose to prominence when the Romans crushed the Selucids and would be the grand power counterweight to Rome for a long time made extensive use of horse archers (using the Parthian shot, ie turning around in the saddle and loosing the arrow over the back of the horse after having turned the horse around) and heavy cataphract armoured cavalry.

The early Roman army made use of landowners who paid for their own equipment - essentially the middle class, divided into three classes - the hastati, the principes and the triarii. The hastati fought as light infantry, with short swords, large shields and javelins, much like the Iberians. It was probably learned from the Samnites, who made use of mobile swordsmen in their wars against Rome. The earliest Roman armies probably fought in a Greek-style hoplite phalanx. The hastati wore what little armour they could afford, while the wealther principes while armed the same way had heavier armour. The triarii consisted of the most affluent, oldest and most veteran Roman landowners and fought with spears. Poorer people fought as skirmishers, mostly with javelins as velites. The upper classes, the equites, fought on horseback as light cavalry and might have been dragoons, as they in several battles where the conditions became too cramped seem to have dismounted to fight on foot. Generally, the quality of the Roman cavalry was not as good as their infantry, and they were regularly bested by other countries' cavalry. The later Roman army after the Marian reforms is made up of the heavy, well-trained and well-armoured infantry in iron armour (lorica segmenta or chainail), iron helmets, large square shields, javelins (pila) and short swords. The Romans were unique in this era by having a formal system to replace the men in the frontline to allow them to rest and always having fresh troops at the front.

The Romans also made extensive use of auxulia (semi-permanent mercenaries) and mercenaries as they saw fit. When fighting the Parthians, they brought large amount of cavalry - Saramatians, Germans, Scythians or Gaul - and large amount of foot missile troops such as Syrian and Cretan archers and Macedonian and Rhodian slingers. When fighting Illyrians or Iberians, who made extensive use of light troops and hit-and-run tactics, the Romans hired light infantry and cavalry to fight them. While the Macedonians made a combined arms army, the Romans made a modular army with a core of legionaries to which the needed auxulia could be be added.

All made extensive use of mercenaries at the time. Illyrian medium infantry, using large oval shields, short swords, some armour and javelins were sought after medium infantry, capable of both skirmishing and fighting in the line.

Dacian (from present-day Romania and Bulgaria) swordsmen and falxmen (using a large reverse blade sabre used either with one or with both hands) were also sought after.

Greek hoplites were still good heavy infantry and were extensively used as mercenaries and garrisons by the Romans after they conquered Greece and made up large part of the Roman armies that fought civil wars in the Greek world.

Thessalian cavalry continued to be an important part of warfare in the Greek world. Sought after as mercenaries, King Pyrrhuss of Epirus had his cavalry sunk en route to Italy and had to fight pretty much without them, which severely hampered his ability to destroy Roman armies thus the costly Pyrrhic victories.

Cretan archers were excellent skirmishers who were said to be able to take on any other archers of the era. Often said to have had a longer range than other archers. These were not restricted to only Crete, as it was more a style of warfare than an ethnic group. Cretan archers seem to have existed over a large part of the Greek world.

Syria procuced very good archers that were also sought after.

Armenia produced horse archers and heavy cavalry that all that had access to liked to hire.

The central regions of Anatolia were the home of Celts who had settled there after raiding through Greece. These Galatians as they were called were common mercenaries, producing renowned armoured swordsmen and scythed chariots.

Macedonian, Rhodian and Balearic slingers were renowned for their skill and ability, and were often hired as mercenaries. The Romans were fond of using these to counter Parthian horse archers as slingers usually had longer range than archers and foot troops were cheaper and could be had in larger numbers than horse archers.

Numidian javelinmen were highly sought after, and the Carthaganians used them as mercenaries extensively, both on foot and on horseback.

Saramatians were a horse people adapt both as horse archers and medium to heavy cavalry and extensively used by the Romans.