You might have a look at the answers to In 1800, France's population was 47 million. Surpassing Russia, Spain, Austria and Prussia. Why historically does France have such a large population and why did it stop booming? They in turn have links to previous answers to similar questions

While I fully support linking to the other answers, I'd just like to note that this isn't just a topic for modern historians, it was a significant political issue at the time. France had just lost the Franco-Prussian War and lost Alsace-Lorraine, and part of the spirit of revanchism was channeled into concerns about the birth rate, especially in comparison to neighboring Germany, which as you noted went from significantly smaller to significantly larger in population.