Indeed there were. The best example would be the crisis of the 3rd century, where endemic coups and civil wars led to 30 emperors ruling in a 50 year period, the Persians, Vandals, Goths and more invaded, there was plague and economic collapse, and both the eastern and western provinces of the empire split off into independent entities because the central government was no longer able to protect them. And yet Rome survived, and indeed experienced a period of renewed growth and prosperity in the 4th century.

u/ChrisTheClassicist discusses these events here

The closest the Roman Empire ever came to collapsing before its gradual decline in the 4th and 5th centuries was during the 2nd Punic War (ca. 218-201 BCE).

The following is an excerpt from one of my essays followed by an explanation of the passage.

The year 221 BCE saw the rise of Hannibal. He continued down the paths laid out before him by his father and brother-in-law by conquering large parts of Iberia. Roman ambassadors sent in 220 BCE to warn Hannibal not to move against the city of Saguntum were ignored and in 219 BCE Hannibal conquered the city. A Roman ultimatum to surrender the city and Hannibal was rejected and in 218 BCE the war officially began.
Hannibal appeared within Italy’s borders earlier than expected, after having traversed the Alps in a dazzling display of courage, and caught the Romans completely off guard.(2) He defeated a Roman army under Publius Scipio at Ticinus and in doing so secured the help of many Gauls in the region.(3) A second Roman army under the command of Sempronius Longus was lured into battle and beaten by Hannibal near Trebia.
By 217 BCE Hannibal had progressed deep into Italian territory and chose to lie in wait for a pursuing army. The resulting battle of Lake Trasimene cost the Romans their only army that could stop Hannibal’s advance deeper into Roman territory. Despite Rome being within striking distance, Hannibal chose to go south to enlist the help of more allies.(4) As a result of the battle at Trasimene, the new Roman commander Quintus Fabius Maximus adopted a new strategy and chose to avoid open confrontation with Hannibal’s army: a strategy much disliked by his soldiers.(5)
Fabius’ unpopularity led to the appointment of new commanders in 216 BCE who adopted a more aggressive approach. An army of 86.000 men was raised to confront Hannibal and bring an end to the war. The resulting battle at Cannae was a colossal military disaster. In the wake of the Roman defeat the loyalty of Rome’s allies started wavering and many of them defected.(6) Despite all these setbacks the Romans were still able to field massive armies, vastly outnumbering Hannibal’s forces, allowing them to prevent the entirety of Italy falling into his hands. However, the Carthaginians continued their expansion until it reached its apex in 212 BCE.

(2) Livy 21.32-38.
(3) Zimmerman 2011, 284.
(4) Zimmerman 2011, 285.
(5) Zimmerman 2011, 285.
(6) Polybius 3.118.

Zimmerman, K. 2011 "Roman Strategy and Aims in the Second Punic War", in: B. D. Hoyos (ed.), A Companion to the Punic Wars, 280-298.

Answer to your question: The first 6 years of the Second Punic War vs Hannibal/Carthage were an absolutely unmitigated disaster for the Romans. The losses they suffered were immense, their enemy had pierced deep into the heart of their empire and was within striking distance of their capital. The military defeats they had suffered during these years rank among the worst in their entire history, with the loss at Cannae being their most infamous defeat. Hannibal had a strong foothold within Roman territory and could have potentially ended Roman hegemony forever if he had had just one or two more such destructive victories.
However, the Roman ability to field large armies was their saving grace. Despite the massive casualties they had suffered they could still field massive armies, preventing Hannibal from conquering the entirety of Italy. By 210 BCE the war finally turned in the favor of the Romans. Scipio invaded Iberia and captured Cartago Nova while in Italy itself Hannibal was having trouble protecting all the new allies he had made. The various cities and states that had defected from the Romans demanded from Hannibal that he protect them. With his army deep in enemy territory, which was a logistical nightmare, and the areas he had to protect growing in size he spread himself too thin. His march had lost its momentum and was stalling out. A reinforcing Carthaginian army was destroyed in 208 BCE which forced Hannibal to flee to the south of Italy from where he would eventually flee Italy to prevent the destruction of his army.
The first few years of the war had severely humbled the Romans. If Hannibal had been able to consolidate his victories more effectively and had not stalled out by playing a more passive approach it is quite possible that the Romans may very well have lost the war before they could gather and organize new armies to replace those lost during the initial conflicts. They came within inches of losing the war, but instead they managed to turn it around and used it to propel their hegemony across the Mediterranean. From then on out their might only grew, as did their reach and territory.