There is substantial variation in this from context to context. The ancient army for which we have the best records is that of the Romans. From the Imperial Period, we have records that the minimum height of a Legionary was 5’5” (165 cm). From skeletons of the volcanic ruins of Pompeii and Herculaneum (both destroyed 79 AD), we have estimates of height for the average adult male (of any occupation) of 5’5.5” to 5’6.5” (166-169cm). There may have been variation in height over time, as the chief source of variation in adult size across time is diet and disease. (For instance, it has been shown (pdf) that children under the age of five during the Chinese Famine of 1959-1961 were one inch (2 cm) shorter than those before and after.) So from period to period and place to place inside Roman territory in their 1000-year history this average may have varied. And it should be noted that the 1st and 2nd Centuries (AD) were actually a period of noted prosperity and stability in daily life in its Italian heartland. So, simply put, Roman Legionaries were often not large men.

As their record of conquest shows, that ultimately mattered little, as success on the battlefield was built more on cooperative action and discipline than individual brute strength alone. This varied from context to context and especially based on the style of warfare. When armies relied less on formation tactics and more on individual combat, size had more opportunity to tell. The armored knights of Medieval Europe were often unusually large for their period. But this occupation was hereditary, and their advantage in size was often more about their wealth and privileged social position from being in the military caste.

In addition, during periods of warlordism the status of warriors was often built as much on combat off of the battlefield. Both individual duels and mock combat (like Medieval tournaments) gave large men an opportunity to demonstrate, and gain wealth and authority from, their puissance. But in general, it was not especially effective for an army to win more battles by having larger and larger men. Those gains were usually dwarfed by having more, and better equipped, soldiers.