Why do we seemingly accept the massive numbers given by Roman sources about their battles in antiquity, but disregard others as being "exaggerations"? What's the evidence that 80,000 Romans did in fact fight ar Cannae, and that this is not just another wild big number Polybius uses for impact?

Quibbling over the size of forces never struck me as a particularly meaningful use of time for historians. There are a few related questions which are meaningful - such as if a battle was won through strength of numbers, or through quality of forces - but they tend to be tangential. But the question of if a force was 100,000 soldiers or half of that is not, on its own, particularly relevant. The debate over size of forces tends to be a proxy for a debate on a deeper issue, the reliability of sources.

To the extent that Roman sources tend to be believed on size of forces, it would be because of the following factors:

The Romans had an exact ordering of their forces, with the legion being a fixed size. This allowed historians a very quick and dirty estimate on the size of a Roman force - even though standard formations are often below strength, it's at least a real number as a starting point, since in most cases it's known how many legions were present in which battles.
In contrast, many other forces in antiquity had no standard formation size; they were often organized by clan or tribe, but that gives no indication of the number of soldiers actually present. (For other forces which did have exact ordering - such as the Mongol tumen - the numbers are consequently more reliable.)
Many Roman sources were themselves military leaders or veterans (Julius Caesar, Josephus, etc.) Other historians did not always have military experience - the Christian and Islamic scholars of the Middle Ages, for instance, typically had no relevant military experience - and so are less reliable in estimating size of forces.
Roman sources are often contemporary sources, and contemporary sources tend to be much more reliable. Roman sources were often not only writing soon after the events occurred, but in many cases - like Caesar - were literally there on the field of battle.

All that said, Roman force estimates tend to be more reliable for Roman forces. Force estimates for opposing forces from Roman sources, while perhaps more reliable than a number pulled out of thin air by a historian far removed from the event, are often considered suspect due to the need to propagandize for the Roman military. Hence, opposing force size estimates are often considered to be inflated.