This is a complex question to answer, and a lot of data relating to the population growth rate after the Civil War is debated. The 1870 Census was controversial in that it was widely accused of under-counting the population, especially in New York City and later, the Southern states that were embroiled in the social turmoil of Reconstruction and military occupation. Logistically, it was very difficult to determine the demography of the former Confederacy. The time after the war was incredibly dynamic, violent, and politically unstable. Because of this, my answer is going to involve conjecture.
What we can reliably say is that population growth rates took a steep decline in the postbellum South. The national population growth in between 1860-70 (as presented by the latter's census) corresponds to 31.45 million in 1860 to 38.56 million in 1870; about a 22.6% increase nationwide average.
In Alabama, the population change from 1860-70 went from ~964,000 to ~997,000. Barely a 3% increase. In Mississippi, the change was about ~791,000 in 1860 to ~828,000 in 1870, a 4.7% increase. Virginia, the most populous Southern state, went from ~1,220,000 to 1,225,000, a 0.41% growth rate (I believe West Virginia's population was later disregarded from this 1860 figure, though I'm not totally certain). Poor old South Carolina saw a change of just 0.27%, from 703,700 to 705,600 over 10 years.
Contrast this with the Northern states. Between 1860 and 1870, New Jersey's population grew by about 35%. Massachusetts by 18.4%. Indiana by 24.5%. Michigan by 58.1%. Pennsylvania by 21.5%. Most Northern states fell into double digit growth, with only a couple exceptions. In the quickly-growing Western states, population growth was as high as the triple digits (well over 600% in Nevada in this 10 year period). The contrast is night and day between the regions. While it is likely that the Census did underestimate Southern population changes to some extent, it's not debatable that the Southern population growth rate was incredibly stagnant in the post-war years.
There are a few contributing factors to this. War was mostly fought in the South, and the rate of troops getting mustered into service in the Confederacy was far more extreme. 6% of Northern white men ages 13-43 died in the war. 18% of Southern whites aged 13-43 died. The Confederate Army mustered somewhere in the realm of 900,000 to 1 million white men, (the total white male population of the entire South was only around 2.8 million). Conscription rates were double in the South what they were in the North, and by the end of the war, the South was scraping the barrel and pushing children as young as 12 onto the battlefield. Somewhere around 30% of Confederate soldiers died in the war. This definitely helped aid the South's population growth depression post-war. In recent years, there's even been statistical analysis done that many Civil War historians have begun to take seriously that suggests we may be far underestimating the total Civil War death toll by 100,000 or even more. Data on the number of slaves and white civilians who were killed still remains hazy. Historian David Blight of Yale has said the figure that approximately 1 in 4 runaway slaves that fled for Union lines or contraband camps during the war died in the process.
Population growth in the North was also heavily supplemented by the 2.1 to 2.2 million immigrants that arrived to the United States between 1861 and 1870. The vast majority of immigrant arrivals came in Northern cities like Boston and New York. With the South being a hotbed of racial tension complete with white supremacist terrorism, military occupation, and destroyed economic infrastructure, most immigrants stayed in the North or moved to the Midwestern interior. The immigration of this decade also skews what the real birth rate might have actually looked like in the North, and comparing it to decades immediately prior or after can be difficult to do with so little reliable data on it.
In 1917, a political scientist named A.B Wolfe set out to try and get an idea of birthrates following the war in the North. He observed official birth registration stats from the state of Massachusetts and observed that the total proportion of white children to the total white population was 2% less in 1870 than 1860, where as it only declined 0.2% between 1850 and 1860. While this is only a brief snapshot of one Northern state, it does show that there has been interest in the post-war population shifts for over a century and that it seems to have resulted in a declining birthrate in many areas.
Zooming out to national population growth rates by decade, we can see the 1860-70 period really stick out.
1830: 33.5%
1840: 32.7%
1850: 35.9%
1860: 35.6%
1870: 22.6%
1880: 28%
There are many minor contributing causes to the population decline (increasing urbanization, lower immigration rates during the war years, etc). But I'd say by far the biggest cause was the war and its effects as well as Reconstruction to some extent. Based on the snippet of data from Massachusetts provided by Wolfe, I don't think it's a stretch to say that birth rates actually declined considerably in the aftermath of the Civil War, the opposite of a baby boom.