I wouldn't say the combat ability of the Roman army really declined at all. In terms of military organisation, while there was the division of the empire's professional soldiery into limitanei (border guards), comitatenses (mobile field troops) and the scolae and palatini (the emperor's household guard regiments), supplemented by foederati (auxiliaries stationed on the frontier) and bucellarii (the private retainers of imperial officials and noblemen), the legion system survived until the early 7th century - the last legion, Legio V Macedonica, founded by Emperor Augustus himself, saw its last action in Egypt in c.640, shortly before the Muslim conquest. Elements of the Marian era legion organisation even survived well into what most people call the "Byzantine" period - centurions commanding units of 100 men are still mentioned in the Taktika of Emperor Leo VI, written in c.900. In terms of size, while the size of the later Roman army remains controversial it was definitely larger than that of the Principate, and Warren Treadgold has plausibly estimated that in 395 there were 645,000 active Roman soldiers. The barbarisation thesis - that the Roman army came to be dominated by German mercenaries who were less well equipped and disciplined fighters with no loyalty to Rome beyond its ability to pay them - has been debunked recently by Hugh Elton; see his "Warfare in Roman Europe, 350 - 425" (1998). Equipment doesn't seem to have declined either but simply changed to meet new needs i.e. lorica segmentata (which had only been the norm for Roman legionaries in the first two centuries AD) and pilae went out by c.300 AD because they were very expensive for army that was increasing in size and were too heavy for an army that was becoming more mobile (only the limitanei stayed in one place). There's no real justification for thinking that mail shirts and ridge helmets were less effective than lorica segmentata and imperial gallic helmets, or that the spatha, oval shield and plumbata were less effective than gladii, rectangular shields and pilae. Finally, the officer corps if anything became more meritocratic in the later empire given that from the time of the late 3rd century soldier emperors (Aurelian and Diocletian) on, efforts were made to break the traditional dominance of the senatorial aristocracy over the military high command and even actively exclude them from it in favour of talented provincials i.e. both Diocletian and Constantius (the father of Constantine) were from relatively humble Illyrian backgrounds, and Emperor Justin, who rose to the throne in 518 through being captain of the imperial guard, was of Danubian peasant background.

​

Therefore it wasn't really military effectiveness that declined. Indeed, as Peter Heather points out in the conclusions to his "The Fall of the Roman Empire" (2005), besides the fluke of Adrianople in 378, which was a defeat that stemmed purely from the arrogance of Emperor Valens in not waiting for Gratian's reinforcements, Rome's 4th century encounters with the barbarians read like a string of Roman victories i.e. the Salian Franks were totally pacified by Emperor Constantine the 330s and the Alemanni by Emperor Julian in 359. The problem was that the Roman Empire, due to its lack of any kind of constitutional safeguards on the emperor's legitimacy, kept throwing up military usurpers in the provinces. These were easily dealt with if there were no major external threats to the empire's security i.e. Diocletian faced 6 different usurpers in his 21 year reign from Carausius in Britain in 286 to Eugenius in Syria in 303, yet none of them troubled him too much. However, this all changed with the migration of the Huns from Mongolia to the Great Hungarian plain, which between the 370s (when the Huns arrived in what is now Ukraine) and the 410s (when they likely came to what is now Hungary) led to the Germanic tribes being pushed into Roman territory. The Goths were the first, in the 370s, and they caused a fair amount of trouble before being co-opted into the Roman army by Theodosius in 392, after which they helped him defeat the usurper Eugenius at the Frigidus in 394. However, under Theodosius' young son Honorius, more barbarian tribes were pushed in as knowledge of the Huns advancement into Europe spread. In 406, the Vandals, Alans and Suebi all crossed the Rhine, while the Roman forces from that region had been pulled into Italy as Stilicho, the magister militum, had to defeat Radagaisus and the Ostrogoths migrating across the Alps to attack Rome. Radagaisus might have been fleeing the Huns, who the rest of his people a generation later were vassals to, though Augustine says it was also his intention to sacrifice the Roman senators to his "demon gods (presumably Odin, Thor, Tyr and the other members of the Germanic pantheon)." Stilicho easily defeated Radagaisus at Fiesole, near Florence, in 406, but was then assassinated by Roman senators who resented his influence. The power struggle for the military high command led to the sack of Rome in 410 by Alaric the Visigoth, and this threw up 6 different military usurpers in the provinces i.e. Constantine III in Britain, who in 409 decided to command all the Roman legions stationed there to come with him to Gaul so he could fight back against the barbarians and take over from the incompetent Honorius. The barbarians, who were now well into Gaul and Spain, were co-opted into these struggles by the various contenders for power and so they took the opportunity to carve out kingdoms for themselves i.e. by 418, the Visigoths had a kingdom at Toulouse, and the Vandals, Alans and Suebi were squabbling over Spain. Subsequent power struggles in which the barbarians were co-opted into led to further problems i.e. the struggle between Aetius and Count Boniface for magister militum under Honorius' successor Valentinian III in the 430s led to the Vandals taking over Africa, the richest province in the Western Empire, as Boniface made the mistake of inviting them over to fight for him. What enabled the barbarians to take over so much of the Roman Empire was thus not so much the ineffectiveness of the Roman army, but rather the internal power struggles going on within the Western Roman Empire.

Would just like to say that even under Augustus the Romans suffered Germanic defeats. Augustus wished to extend the empire to the Elbe River and was never able to gain a strong enough foothold in the region outside of raiding.

Additionally, Augustus is said to have famously had his facade of success and confidence shattered when 3 Roman Legions were defeated in the Battle of the Teutoberg Forest

Read more about it here: https://www.smithsonianmag.com/history/the-ambush-that-changed-history-72636736/