I recently started reading Caesar's Gallic wars, and after reading about the battle of Bibracte I looked at the wikipedia page to see if there was any more information. Caesar records the number of Helvetii as 368,000 (a figure he got from their own records which he looted) - but the wikipedia page lists three sources for the number of the Helvetii

Caesar:
368,000:
90,000 warriors
278,000 non-combatants
Orosius:
157,000 people
Modern estimates
20,000 people, including 12,000 warriors

This isn't the only time I've come across something like this. I loved reading Herodotus' histories last year, and there's great detail about the number of people who marched into Greece with Xerxes. Herodotus seems very aware that the size of the army beggars belief. He talks of how they drank rivers dry and impoverished every town they came to. He even details the means by which they counted the number of people there were (by having them stand in turn in a circle of known capacity). Yet Wikipedia confidently tells me this:

The numbers of troops that Xerxes mustered for the second invasion of Greece have been the subject of endless dispute, because the numbers given in ancient sources are very large indeed. Herodotus claimed that there were, in total, 2.5 million military personnel, accompanied by an equivalent number of support personnel.The poet Simonides, who was a contemporary, talks of four million; Ctesias, based on Persian records, gave 800,000 as the total number of the army (without the support personnel) that was assembled by Xerxes. While it has been suggested that Herodotus or his sources had access to official Persian Empire records of the forces involved in the expedition, modern scholars tend to reject these figures based on knowledge of the Persian military systems, their logistical capabilities, the Greek countryside, and supplies available along the army's route.

Modern scholars thus generally attribute the numbers given in the ancient sources to the result of miscalculations or exaggerations on the part of the victors, or disinformation by the Persians in the run up to the war. The topic has been hotly debated but the modern consensus revolves around the figure of 200,000 or 300,000–500,000.

Why would modern scholars be so sure that historical sources are over-representing the numbers present? There are three listed contemporary sources, all of which say 800,000 or higher - yet scholars opt for at most 500,000 - a massively lower number! The sources even explain how they came to these numbers and provide accounts believably consistent with the numbers listed. It seems to me that scholars are assuming a lot to think they know better than contemporary writers about something that happened so long ago. Or is there a really good argument otherwise?