I know this probably sounds like a simple question but despite all we know about the Roman army, it's something we may never be able to fully explain
I would like to start by saying that throughout the Roman state's history, the effectiveness of its army fluctuated greatly. That being said, there is a certainly a general decline in the effectiveness of the Roman military apparatus after the Antonine dynasty. One could also argue the Roman Republic's unraveling also decreased military efficacy but that's getting off topic. Despite many, many, examples of Roman military success post Marcus Aurelius, the cracks have begun to form and will never truly be repaired. In addition to the decline in the offensive capability of the Roman military, the state adopted a different outlook to warfare and diplomacy. The Roman Republican elite, and to a certain extent the Principate's rulers, were generally focused on winning glory, fame and riches through offensive conquest. The focus of the Roman elite slowly transformed into a desire to stay in power and protect the preeminent status of the Roman state over its peers. While this may not seem like a major change, the effects of transitioning the Roman military from a primarily offensive one, to a primarily defensive one, had systemic impacts throughout the Roman military and the Roman state as a whole.
Let's start with early Roman state's relationship to it's military. Rome undoubtedly possessed one of the greatest conquest armies of all time. The Roman state's ability to field large armies with good discipline, led by competent commanders, who could win pitched battles and sieges consistently, enabled an incredible amount of expansion throughout the Republican period. This, coupled with a ruling apparatus focused on improving Rome's position, created a state with the will and ability to expand quickly. The Roman Republic's light touch governing style was extremely effective at winning over local elites after the conquest. The Roman state demanded very little of local elites post-conquest while the Roman system was extremely politically, culturally, and economically desirable to those local elites. This contrasts sharply with the later Roman or Byzantine state. Even where the Byzantines were successful in their conquests, they were not able to turn these into long term gains. In part, high taxes and religious conversions were not appreciated by the common folk, and Byzantine administration chafed local elites. Regardless of all the reasoning, the outcome is clear: the Roman state during the Republic and Principate was more willing and able to conquer territory than during the later Byzantine period.
Next up, the army itself. One of the most important features of the early Roman Army was its ability to beat the Macedonian Phalanx. Whether this is down to tactics, equipment, superiority of non-commissioned officers, or a host of other factors, the facts stands that in pitched battles like Magnesia or Cynoscephalae, the Roman system just worked. In just a few victories with very few relative losses, the Roman Republic was able to undo almost all of Alexander's conquest and conquer the entirety of the Eastern Mediterranean. While the Romans had other successes during this period, they were much more hard fought. Caesar's Gallic campaign, the Conquest of Iberia, and the Punic Wars were all eventual victories but pressed the Roman state on many levels. Even relatively minor powers like the Cimbri, Teutones, the Parthians at Carrhae, and Germanic tribes were able to inflict severe losses on the Romans. As time goes on the Romans find themselves having wiped out all the easily defeated and assimilated states around the Mediterranean, and faced with a variety of foes that aren't interested in being Roman, and are much more effective at defeating Roman armies. In the late Roman and Byzantine periods, defeating a neighboring state meant that a new one would move into the vacuum creating a net neutral result and often a net negative if the new power turned out more aggressive or more powerful than the last (Parthia > Sassanid Persia, Avars > Bulgars, Ostrogoths > Lombards, Sassanid Persia > the Arab Empires etc.) The Romans started their rise to power more organized and militarily effective than their neighbors. After importing Greek culture they were also seen as a leading cultural force. As time goes on, it seems that other states catch up and the Romans aren't able to win simply because they can put more soldiers in the field for longer. In the late Empire, many of Rome's opponents served in the Roman army and had Roman training, tactics, and equipment. Without advantages in technology, tactics, logistics and organization, the Roman army was still effective, it just wasn't as overwhelming as it had been in the past. In the late Empire and through the Byzantine period, the Romans often find themselves faced by very similarly effective militaries and win or lose on generalship more often than not. In fact, the Romans start finding themselves outmatched in terms of tactics and equipment, and several Byzantine military manuals mention the need to copy the war making methods of other peoples as they give the Romans the best chance of victory. That being said, adopting tactics and equipment was something done by the early Romans as well and a big reason for their success. My point is that military innovation seems to shift from a Roman led military status quo, to a more even one between whatever contemporary powers existed at the time.
edit for grammar
1/2