The Roman army that fought at Cannae was at least 40,000 strong in the assessments I've read, and after it was wiped out, Rome was able to raise further large armies to fight in Italia and Hispania simultaneously not too long afterwards (it was also, as far as I'm aware, composed almost entirely of Romans and socii).
By contrast, Belisarius reconquered Africa and (some of) Italy with armies of less than 10,000 men, including a good number of mercenaries recruited from outside the empire, and who were hired at the expense of keeping up payment to limitanei. Despite the loss of the West, the Roman Empire at the start of Justinian's reign still controlled a lot more territory than the Republic at the time of the Second Punic War, so why were they only able to raise much smaller armies?
EDIT: Thinking about this after posting it, I realise that a useful answer might have to be very broad and vague indeed - something like 'here's the social structure of the Republic in 200 BCE, here's the social structure of the Empire in 500 CE - compare'. I suppose what I'm wondering is whether there are some broad reasons why the two states differed in their ability to raise armies, such as population density, the attractiveness of a military career to young men, the ability to take a higher or lower percentage of national revenue in taxes without leading to resentment and unrest, etc.
I cannot say anything about the Roman Republic, but it is I think worth placing Justinian's smaller armies into their context. His empire had a standing military force of c.300,000 men, which, in my unqualified opinion, must have outnumbered the total military force the Romans had at any one time during the Punic Wars. Justinian's generals could never use all of these men in one place of course, since half were stationed at the border, which left 150,000 men in the field armies. There were 6 of them before Justinian began his conquests, two near Constantinople, one each in Illyricum, Thrace, Armenia, and Mesopotamia, which meant that each army had c.25,000 men - a reasonable figure as our sources for the major battles in the sixth century say that the Roman side often had c.20,000 men.
Certain campaigns, such as the Vandalic War, did involve a smaller number of soldiers, as you noted, but it is important to remember that this campaign was an unpopular one and that Justinian's court largely disagreed with the decision to invade. Sending an overwhelming force, or even an entire field army, was therefore never on the agenda. The whole campaign was also dependent on uprisings in Sardinia and Libya, which perhaps suggests that a smaller force was sent on the assumption that they would gain further support in North Africa. It is also important to note that although there was peace in the east, it did not necessarily mean that the frontier could be left undefended. Nor could troops be denuded from the Balkans, which remained unstable. A small expedition is therefore, I think, very understandable given the circumstances, especially as the Vandalic War was by no means a life-or-death struggle. When the threat to the empire was more acute, such as in the seventh century, larger armies are mentioned in the sources - I wrote about some of the numbers involved here.
In the Gothic War it became harder still to muster up larger armies, as the Persians reopened the war in the east in 540. Belisarius did initially invade Italy with a smaller force, but a parallel invasion of Italy through Pannonia happened at the same time, so the initial invasion actually involved a lot more people. In time less and less resources could be devoted to the west, simply because of military pressures elsewhere, but like in the seventh century, when there was a will, there was a way, and larger forces were mustered by Justinian, as seen in the climactic Battle of Busta Gallorum in 552, where the Romans fielded 20,000 men.
Finally, it is worth pointing out that Justinian's army was mostly made up of voluntary recruits (though manpower became more of an issue as time went on and conscription certainly did occur on occasion), so manpower was not an issue. The biggest structural factor, I think (and remember that I know nothing about the Republican army), is that Roman soldiers in the sixth century were professionals who formed a standing army, whereas Roman soldiers of the previous period were levied for a certain campaign and expected to provide their own equipment? In the link above I was also rather sceptical about the size of large armies, but I'm not sure if the same point can be made for Classical sources, so I am eager to hear from some ancient historians as well on this issue :) My personal view on this issue is explained here, featuring also the helpful input of /u/Iphikrates and /u/PapiriusCursor.
I can't really discuss this further as I am not a military historian, but hopefully a proper Roman specialist will appear and provide a suitable answer to cover what I left out. I hope this was still useful though!
I was wondering the same thing. I watched a multi part video on YouTube about the first and second Punic wars and it seemed each time Rome was defeated, they would raise an army almost twice the size each time. I guess a follow up question would be, were all those men fully trained in combat? If so, how was that logistically done? Was it similar to military drafts of our time?
Though I can't say anything about the Justinian era I can tell a bit about the republican era.
In most of the republican era the Roman army was basicly a militia. Any Roman citizen which had a certain amount of income would join the Roman army if they were called upon for a war. Most of the Romans who had enough income would not hesitate to join the army as it was seen as an honor to fight and it could be very lucrative if you got lucky and get lots of money from the spoils of victory. This allowed the Romans to have field bigger armies than would be possible with just mercanaries or professional soldiers. This method was commonly used in both in Greece and Italy though Hannibal relied on mercanaries for his campaigns in Italy, which losses for him were a pretty big problem while Romans could fill their ranks by recruiting more eager citizens.
Probably more important for the amount of men the Romans could raise was the alliance system they would use. Rome often forced alliance on their beaten foes. This alliance would entail that the ally would have to supply soldiers for Roman wars (which were often militias like the Roman armies at the time) and would have to maintain these at their own expense but in turn they would be allowed to join in the spoils of victory when a war was won. The allies were very loyal due to this very favourable alliance. Even in the Punic wars many allies stayed loyal to Rome due to these favourable terms, because many were afraid that if they would help Hannibal win he might impose harder terms on them like taxation.
At the beginning of the Second Punic war most of Italy was either in direct control of Rome or was in an alliance with Rome, as such they had the capabilities to raise very large parts of the Italian population for their campaign. Even though many of the battles were lost in the end the Romans could always recruit more soldiers from either their allies or their own citizens, while Hannibal who was mostly reliant on mercanaries which were hard to replace.
These are probably not all the factors which were important as I am not yet specialised in any form of history as I am still studying. If anyone who is more proficient in this part of history would be able to add to this that would be much apreciated.
My sources are the lectures I had about Rome and the book I had to read for this which is sadly in Dutch which is called : "De Oudheid Grieken en Romeinen in de context van de wereldgeschiedenis" by F.G. Naerebout and H.W. Singor. I sadly do not know if there is a translation of this book.
Sorry if there may be any grammar or spelling mistakes I really wanted to answer the question but had to do it quickly. I'll try and clean it up tomorrow.