The Roman Empire was a very big place with a very diverse group of peoples, so we are likely to find pretty wide variations between people of different regions, periods and social classes. That said, Geoffrey Kron went through 49 separate studies on the remains of adult males on the Italian peninsula which dated from 500 BC to 500 AD. By synthesising all the results he found an average height of 168.3 cm (5’ 6.4”). He cites another study which found a standard deviation of less than 7.1 centimetres, which gives you an idea of the variation in heights.

Studies on female remains from Herculaneum have found average heights of 155.2 cm (5’ 1.1”) and 151.7 cm (4’ 11.7”). This is a smaller sample study, but it gives a bit of an idea on the height of women.

Typically, what they do to find these heights is to measure different bones and extrapolate into heights.

Sources: Geoffrey Kron’s Anthropometry, Physical Anthropology, and Ancient Health, Bisel & Bisel 2002, Capasso 2001