There seem to be quite a lot of battles like the battle of Carrhae or Cannae, where tens of thousands of roman soldiers were killed with comparetively minimal losses on the opposing sides. How was it that the Empire remained so powerful despite of this?
Every campaign had some unique features, but the overarching answer during the republic was manpower. At the outset of the second punic war, Rome had a manpower reserve of roughly 700,000. Carthage could not come anywhere near this number, and relied heavily on mercenaries. Roman consular armies in the mid-republic era typically consisted of two roman legions and two allied legions, the distinction being the towns from which the men were drawn. These armies were roughly 20,000 strong. At the battle of the Trebia, for instance, Hannibal defeated two consular armies and destroyed as much as 75% of the force. The battle at Lake Trasimene again wiped out as many as 30,000 Romans. Finally at Cannae, a super-army of 80,000 was almost completely destroyed (along with a significant portion of the roman senate, and the dictator). These battles each effectively wiped out Rome's fighting force.
However, if you add these numbers up, you'll see that the Romans still had more than a half million men upon which it could call to field an army. The problem was the time it took to actually form new legions, equip and train them and such. Hannibal made what many claim was a major blunder by marching not to Rome after the battle of Cannae, but to Capua. Capua welcomed him, whereas Rome was fortified and Hannibal probably could not afford a protracted siege. This is a subject of much debate, and I won't take a stance. However, by the next fighting season, Rome could deploy a new army, and Fabius "the delayer" essentially avoided fighting hannibal for an entire season. Hannibal had little ability to replenish his forces from Carthage and Spain, and instead had to rely on Gallic and Italian allies. What followed was a decade long meat grinder campaign up and down Southern Italy. Rome meanwhile had forces deployed in Spain which enjoyed success for 8 years before a disastrous defeat that resulted in the deaths of two Scipios. They also had forces deployed in Sicily (the forces who survived Cannae and were punished by being sent to Sicily and not allowed to return to Italy).
So at least during the Second Punic War, Rome was able to outlast, was probably aided by a small amount of luck from a bad decision, and could practically discard legions it didn't like as punishment (to say nothing of decimation). Some other interesting points of note here are that Rome actually allowed a slave army to be raised in desperation, and it performed fairly well. Rome also sought private donations from Patricians to help fund the war effort.
This only covers two decades of Roman history, but the same concept holds across the history of the Republic and Empire.
Most of the numbers come from Livy, whose numbers I tend to feel are inflated and fantastic, and Polybius, who generally grades as a fairly reliable historian.