Recently I was reading into this topic a bit, freely on my own time, and I remembered seeing somewhere that one of Madison's main oppositional critics of the Federalist Party, whose name unfortunately also escapes me at this time, decried Madison's claims, saying that Madison and his warhawks had grossly over-inflated the numbers by the thousands to justify their desire for war.
Furthermore, I can recall reading somewhere that the accurate numbers were perhaps only in the low, low hundreds at most. Is this true? I am also presently struggling to find the data I need.
The only numbers I can find from the NPS, and government archives suggest that around 9000 - 10,000 impressed sailors claimed American citizenship.
It is likely that the actual number is lower because it was relatively common for Royal Navy sailors to go AWOL for bouts with the American merchant marine, or outright desert. The British (and Americans for that matter) used press gangs to fill Navy man shortages, and more than often aimed at men with "seafaring habits".
Back in the early 19th century there was very little separating Americans from the British. There is very little evidence of a major accent diversion at that point, and the two were very easily confused for one another. In a modern context it is not dissimilar to how English Canadians often get confused for Americans. So this furthermore added to the ambiguity of the nationality of the sailor in question.