Firstly, the Roman army did not become smaller in absolute terms.

By the height of the Punic wars in 212 BC, Rome managed to mobilise perhaps as many as 100.000 soldiers in 25 legions.

In Augustus' day, the professional Roman army was around 300.000 strong.

In the 4th century in the days of the Tetrarchy and after, it is less clear how big the army was but most authors assume it had been expanded significantly in size. It may have been as much as twice as large as the army of the Principate. (Though I personally suspect the increase was not quite that big.)

Secondly, field armies did not shrink very much either: the "average" Consular Army in the days of the late Republic consisted of 2 legions and 2 alae of Italian allies, for a total of some 20.000 men. Larger armies were fielded on some occasions, but those were exceptional. In the late empire, we see Roman armies of a similar order of magnitude, from the battle of Adrianople to the battle of the Frigidus. And we still see occasional mention of much bigger forces. (Zosimus has Julian's invasion force of his Persian campaign number 60.000 - 90.000 men)

Now, proportionally to the size of the empire, the mobilisation rate of the army did decline significantly. And the army became less inclined to fight pitched battles. But these changes reflect the different nature of a professional army defending a large empire versus a semi-professional militia army being mustered from the entire population.

Roman manpower is a whole different discussion, but suffice it to say that scholars no longer believe that the empire was depopulated very significantly in late antiquity.

I go into much more detail on the how and why the Roman army changed in this post:

https://old.reddit.com/r/AskHistorians/comments/fgxsty/why_was_the_late_roman_army_so_much_worse_at/