There's a fun question! Why didn't the Romans maintain that sense of "stiff upper lip, do it for your country?" The population of Imperial Rome was absolutely larger than that of Rome a few hundred years earlier, so why was there such an issue with manpower? Or, mayhaps, why was it considered to be such a tragedy? Or both!

So for context, in 9 CE, there's a common view that the Roman world was peaceful, prosperous, and all was a flourishing golden age under the Pax Romana. New trade routes were opening up left and right, with huge amounts of income and luxuries coming in from places as distant as Britain and India and beyond. The Roman provinces were at peace, and all was well.

Well. Except for that one time. In 6 CE, as a force was gathering to move against the Germans, a general muster was held to gather auxiliaries to the Roman banner. One group of these auxiliaries, upon being mustered, instantly rebelled. This was Illyria, the region right across from the back part of the Italian boot. Illyria had been well known in the day for being the land of pirates, and they had raided the heck outta the Italian coast. Now, it was a Roman province, and the rebellion here was somewhat out of the blue. The army they raised quickly overwhelmed local garrisons, with some (probably exaggerated) reports saying that their numbers swelled to 800,000 men. The uprising immediately panicked Italy, with Augustus being warned that they could be at the gates of Rome within 10 days. He immediately put emergency procedures in place, buying ten thousand slaves, immediately freeing them, and drafted them into a defense force. 100,000 troops were sent into Illyria (though half were quickly returned to their original posts when Tiberius realized that he had more men than he needed), and for the next three years, the rebellion was crushed, slowly but surely.

It was considered a near catastrophe, barely averted. Augustus himself, no longer a young man by any means, was thrown into a panic. By the time the rebellion was quashed, the infrastructure of the region had been devastated, at least one veteran colony was wiped out entirely, and thousands of Roman civilians in the area had been massacred outright. It was, in short, a disaster, especially for a man who had prided himself on bringing peace and stability to the Italian peninsula.

It was finally cleared in 9 CE. Most of the legions along the border had been pulled south to deal with the rebellion, leaving a skeleton garrison of three legions to guard Germania. Their commander was a well-known, extremely experienced man, considered in Roman circles to be a highly competent administrator. And just as the embers of the revolt in Illyria were being quenched, news reached Rome that the border of Germania, a place which had been experiencing near-constant turmoil, conflict, uprisings, and invasions for the past 4 decades, which was consequently one of the most well-defended in the entire Empire, which was the thin barrier between the now-prosperous and essential province of Gaul, and which was only a couple of short weeks away from Italy...the legions there had been utterly annihilated. The news reached Rome, and as you can imagine, the people there instantly panicked. News reaching the city meant that the terrifying German barbarians could be right on its heels, crossing past the Alps and ravaging Italy, to say nothing of Gaul. Augustus realized all of the implications - including the fact that, where his (adopted) father had won Gaul for Rome, he might be known as the man who lost it.

Again, perception is key. These things did not happen, since the Germans themselves were nowhere near organized enough to mount such a campaign. But to the Romans, who had been given - for the first time in a century or so - a stable, peaceful, thriving country, it was objectively terrifying. The people in Rome were generally not soldiers, and most of them had no interest in serving in the military. They were stonecutters or construction workers or librarians - not swordsmen. Many recruits for the legions could be found where soldiers were, like veteran colonies. They were terrified, as any large body of civilians would be, and the idea of the invasion was possibly worse than an invasion itself. Germany became a land non grata, a place where no legion wanted to go, let alone be stationed. There was a punitive expedition by Germanicus to restore Roman honour, but that idea of terror stuck in the Roman psyche quite firmly. New legions could be raised, sure, but those things took time in a period where men were no longer required to serve.

Contrast that to the height of the Punic Wars, when it seems that army after army was crushed by Hannibal as he led his armies through Italy. The differences here are quite stark: not only was Hannibal parading around the Italian peninsula itself, with an army literally marching distance from the city of Rome, but the people themselves were far more accommodating to the idea of being in the army. That service was not going to take 20 years of their life, but one campaign. They - or their fathers - had been regularly serving in the military, and such a military tradition was firmly ingrained. Over a couple of centuries, though, things change. The manpower in literal numbers was certainly there, but the Roman world was a vastly different place. The military had fundamentally changed, the situation was absolutely different, and the danger was no imaginary one, but a real one.

TL;DR: Context. Hope that helps out a bit, let me know if you need more!