At the very least they would face ridicule and derision at home, such as what Gnaeus Cornelius Scipio faced when he lost a fleet to the Cathaginians in the First Punic War. He earned himself the cognomen of Asina', 'ass'. On the other hand, Scipio was elected consul later, so the shame didn't end his career.

In reality though, very few Roman generals escaped with their lives from crushing defeats. In contrast to Athens' democracy where defeats, real or imagined, would result in trials and ostracism, this didn't really take place very often in Rome. Rome itself was a strange mix of some of the democratic ideals of Athens and the militarism and honour of Sparta. The survivors of Cannae for example were assigned to Sicily for the remainder of the Second Punic War, a disgraceful assignment that took them out of further fighting in the war. The leaders of the Battle of the Caudine Forks, perhaps Rome's most humiliating defeat, offered to give their own lives up to allow Rome to renege on the treaty they were forced to sign to save their army (if Livy is to believed) demonstrating that honour was as important as life.

In general though, a military leader in Rome was also a political leader, and thus the consequences for failure as the former generally meant your future as one of the later was also pretty much ended.

We have a very good example for 105 BCE.

Quintus Servilius Caepio actions were considered the cause for the crushing defeat at the battle of Arausio by the Cimbri and the Teutones. 80.000 roman soldiers and 40.000 camp followers lost their life.

It seems that the defeat was caused by his unwillingness to collaborate with his commander Gnaeus Mallius Maximus (Caepio was old nobility, Maximus was a "new man").

He was stripped of his citizenship, his proprieties confiscated and he died in exile.

The Cimbri and Teutones were ultimately defeated by Marius.

Source: Livy's Periochae, book 67.

By the way, Caepio was the grand father of Servilia, the lover of Caesar and the mother of Brutus. Marius, on the other hand, had married Julia, the aunt of Caesar.

I always find it a bit strange how all the major figures of classic Rome were related to each other.