This is something that has always stood out to me. Most of the Roman emperors of the first two centuries seem to have had relatively few children, at least compared to the large family sizes that would have been common in those days. Looking at the first ten emperors (excluding the three from the post-Nero interregnum):
After this Emperors start to regularly have more kids.
All in all, it seems like the early Roman Emperors had a pretty low fertility rate, especially since many of the listed kids died in infancy. This is even more confusing to me because of the high importance of succession.
So why did early Roman Emperors not have a lot of kids?
I think it might be fair to answer this in two ways.
First, to question the premise a bit.
One caveat: these are the issue we know of, overwhelmingly just the legitimate children. But let’s take those numbers at face value.
But also, let’s consider the numbers. You stop right before Antoninus Pius, who had 4 children, and who was followed by Marcus Aurelius, who had 14, far and away the most prolific of all. Lucius Verus, his co-emperor, had 3. But three people could be down to an expected level of random clustering, so we need to look at a larger sample.
Of the next 37 (depending on how they are counted), the original pattern continued: the most common number of children was zero, and only three of them had as high as three, as far as we know (Papienus, Gallienus, Carus).
Of the remaining 25 of the unified empire, as far as we know, Maximian had 3, Constantius I had 4, Constantine the Great had 6, Valentinian had 5 and Theodosius had 5. Magnus Maximus, if we include him, also had 3. The rest all had at most 2, the most common number still being none.
There honestly doesn’t seem to be anything overwhelming about this distribution statistically, though fitting a line does show an upward trend. I’ll do some more analysis in a bit if I get time, but even then we have some confounding variables, and a low sample: a fairly small list of particular men with particular circumstances. But there are a few factors to keep track of:
Many of these were military men, especially during the crisis of the 3rd century. Some spent most of their adult lives campaigning and this gave less opportunity for procreation (at least the ‘legitimate’, recorded sort).
Many did not live particularly long lives, and were prone to assassination. It’s fair to note that it’s only some of the most long-reigning emperors that had a larger number of children. This could be connected to which reigns saw the most stability, and even the effect of power.
Many differed drastically by culture, not just eta: Constantius I had an Illyrian background, and it’s quite plausible - if speculative - that this had an effect on him and his son. Some emperors were of Punic (the Severans) or Syrian (Philip and his son) descent. Likewise, some were Christian, some were pagan. All were to some degree of course Romanised, but it’s very hard to disentangle these other matters and assume that they represent a cross section of Roman Empire of their time
The low sample means massive individual variation is possible: your list stops at Hadrian, who was almost certainly what would today be considered gay. His adoptive son has 4 and his adoptive son had 14. Does this show a grand historical trend? Or just that one individual was gay, and another right after him, to be blunt, had a lot of sex with his wife?
Succession was not managed the same way as in most monarchies we think of today, and heirs were often chosen from other relatives or by adoption. So there was not quite the same pressing need to continue the line that we see in medieval Europe, and in fact it was precisely because being the emperor’s first son did not imply succession that several emperors took pains to declare their sons co-emperors in their own lifetimes.
There is a bigger question at play, where some discussion does possibly support your hypothesis:
The background fertility rate of Ancient Rome in the late Republic and Early Empire was low.
Archeological studies estimating population growth over time and factoring death rate seem to support a low birth rate, as does awareness of Augustus’ famous tax laws to encourage marriage and childbirth (despite only having one of his own), the Lex Julia de Maritandis Ordinibus, framed (arguably similarly, but unlike today) as punishing the single and childless with an extra tax, rather than providing breaks to the married and parents.
This is a frequently discussed topic, with one interesting example here. The possible reasons given in that linked paper include:
Contraception and abortion were common, becoming less so in the Christian period.
Lead vessels and ‘plumbing’ (in the oldest sense) led to lead poisoning, which can affect fertility.
Specifically among powerful families, it was popular to marry single heiresses so as to inherit their parents’ fortunes whole, and this led to a sexual selection towards families who had low fertility rates.
The Romans took hot baths. Very hot baths can affect male fertility by killing sperm beyond a certain temperature. Though widespread throughout the Empire, this may have affected those of Roman origin rather more than those from other cultural backgrounds.
TL;DR: I’m not convinced that this trend is a strong trend. The greater number of emperors with more children in the Dominate than Principate could be a secondary effect based on other issues, especially out of such a small sample. But there is certainly discussion that early imperial Rome had a lower fertility rate, which may have interacted with the later Christian and more ethnically diverse emperors.
I don't have an answer but can I add a follow up question? I'm reading Marcus Aurelius' Meditations and just learned that he basically became emperor by adoption, as was his uncle Antoninus Pius before him. How common was this succession by adoption and is it related in any way to OP's question about the other emperor's natural children?