Diodorus Siculus claims that in the immediate aftermath of the defeat at Cannae the Romans basically thought it was over, the elders opened the city gates of Rome and waited for Hannibal to come, but he never did. (Diodorus Siculus, 25.19)

In addition, the defeat caused some Roman allies (the city of Capua among others) to defect to Hannibal's side, this then led to the Romans feeling they could no longer trust a lot of their allies and subsequently garrisoning cities with their forces to prevent desertion. (Livy 23)

It would probably be best to describe the Romans as 'spooked' following Cannae, though the inability of Hannibal to follow up on his victories or capture heavily fortified Roman cities meant they were not in a losing mindset and remained fully committed to the war.

Diodorus Siculus, The Library of History

Livy, The History of Rome

J. F. Lazenby, Hannibal's War

[This started out as a response to /u/mtmdog on the reliability of Diodorus Siculus, but I ended up answering the main question so I'm posting it as a top level comment now.]

/u/mtmdog says (following Diodorus Siculus' account of events):

the elders opened the city gates of Rome and waited for Hannibal to come

It's not so much that they opened the city gates, but that the city lay open - it was in grave danger, exposed to attack, defenseless. Diodorus Siculus says the elders waited at the gates to hear news from passers-by. The gates are a pretty logical place to hear news from outsiders. Here's the relevant sentence in the Loeb translation:

At this time Rome, when absolutely all its men were lost, stood wide open for many days, and the elders sat before its gates, bewailing that most grievous calamity and asking those who passed by whether none at all was left alive.

This is what Diodorus Siculus says happened, but it should be taken with a grain or two of salt. The entire section reads like a bad historical novel, with the battle itself taking place during an earthquake and meteor shower, with Roman women responding by weeping and 'consorting' with slaves and barbarians, with the elders sitting before the gates, and with Hannibal not taking advantage of the situation because of "drinking and soft living".

Now if this source would be our only source regarding the event we'd have huge debates on what actually happened and which parts are reliable and which are not. We'd establish that there were survivors and some of them did get to Rome quite quickly, so there would be no need to wonder "whether none at all was left alive", and having established that his is an overstatement, we'd argue over how much of an overstatement it would be. None of that is needed however, as for this period we have the texts of Polybius and Livy.

Polybius on the whole is pretty reliable - straightforward and matter-of-factly; he was a very decent historian, certainly for that day and age. He describes events in Rome immediately after Cannae as follows:

The Romans on their part owing to this defeat at once abandoned all hope of retaining their supremacy in Italy, and were in the greatest fear about their own safety and that of Rome, expecting Hannibal every moment to appear. It seemed indeed as if Fortune were taking part against them in their struggle with adversity and meant to fill the cup to overflowing; for but a few days afterwards, while the city was yet panic-stricken, the commander they had sent to Cisalpine Gaul was surprised by the Celts in an ambush and he and his force utterly destroyed. (III.118)

The Romans got their act together however:

Yet the Senate neglected no means in its power, but exhorted and encouraged the populace, strengthened the defences of the city, and deliberated on the situation with manly coolness. And subsequent events made this manifest. For though the Romans were now incontestably beaten and their military reputation shattered, yet by the peculiar virtues of their constitution and by wise counsel they not only recovered their supremacy in Italy and afterwards defeated the Carthaginians, but in a few years made themselves masters of the whole world. (III.118)

Polybius does have some issues of his own, and his account may be a bit biased towards (parts of) the Roman elite, but I'd take his description of events over Diodorus Siculus' tabloid story any day.

Livy is more of a mixed bag than Polybius (mostly because of the various sources he has used) but on the whole his history of Rome is still quite good. He goes into more detail what happened in Rome after Cannae. The praetors in Rome called a meeting of the Senate; they organised the gathering of news and intelligence; they stationed guards at the gates to prevent people from fleeing in panic; they forbade all public displays of grief. Wives and mothers were instructed to wait for news of their loved ones at home. (AUC XXII.55-56)

All of this is at odds with Diodorus Siculus' account of events. Instead of "sitting outside the gates waiting for news" there was an organised effort; instead of wives sleeping with "slaves and barbarians" there were efforts to calm them (and others) down. From both Polybius and Livy it appears that after a very brief initial panic the reaction in Rome was one of urgency and determination.