Kyle Harper's Slavery in the Late Roman World (2011) provides estimates for Roman slaveholding between 275-425 AD. He argues that Roman slavery was at its greatest volume during this time, with millions of people enslaved (about 10% of the population).

Situations could vary broadly. 50% of all slaves lived in what Harper calls Bourgeois or agricultural households with groups of 1 to 5 slaves. 35% of all slaves lived in elite households in core areas of the empire, living in groups of 20 or more slaves. 10% of all slaves lived in elite households in peripheral areas with about 6 slaves each. 3% of all slaves lived on the massive estates of the most elite (only about 600 households in all), in groups of 100s or even 1000s. These large estates averaged about 250 slaves.

I'll be interested to hear how this compares to 19c. America