The world population has increased from ~200M in 1 AD to ~900M in 1800, which is around a 0.08% growth per year. From what I know and have read online, the average fertility was quite high, somewhere above 5 (I do apologize for generalizing across the whole history in this section, but it is my impression that this number didn't vary massively between different times and places before the Industrial Revolution). Also, as far as I know, the infant mortality before modern medicine was around 30-45%, so an average woman is probably going to have more than two children survive to adulthood. The average life expectancy at birth in most places hovered around 30-40 years, so taking into account infant mortality it should rise by quite a bit for those who survived to adulthood.
Given those rough numbers, we "should" see, on average, several children per family surviving to adulthood, each having several children of their own and so on, which would have led to a rapid increase of population; so why didn't it happen?
The first thing that comes to mind would be wars and accompanying famines and illnesses; however, the destruction of the two world wars and Taiping Rebellion has not had a significant long-term impact on the populations of their time - e.g. Japan after WW2 has recovered their pre-war population in less than a decade; even accounting for significant infant and maternal mortality improvements in the 20th century, I can't imagine historical wars being worse than that (leaving aside "small" local exceptions like Genghis Khan and Black Death).
This is a tough question to address due the multitude of inferences you’ve made, as well as the huge timeframe (most of recorded history). I can speak to a few of your points, though.
You mention wars “and the accompanying famines and diseases” in your last paragraph but you don’t address famines and diseases by themselves. Remember that, for most of history, most societies were purely agricultural, and that they struggled to produce even a paltry surplus of food. Famines were extremely devastating to the livelihood of a population, regularly claiming millions of lives. China regularly saw peasant revolts relating to food shortages, and the Great Famines of the 14th century crippled Europe.
Diseases were also incredibly devastating. The Black Plague brought western civilization to a standstill, claiming over 100 million lives according to most sources. Some 800 years before that, the plague of Justinian claimed over 20 million, which crippled Europe and prevented the resurgence of the Roman Empire.
Also remember that the infant mortality rate does not address the types of lives people lived. The vast majority of people before the 19th century were peasant farmers who lived very difficult lives. They had poor nutrition and worked every day to make even a semblance of “ends meet.” People were unhealthy and the development of medicine was hampered by superstition and the lack of scientific methods and implements. Remember that we knew hardly anything about bacteria or the spread of disease until Pasteur in the 19th century.
The ability of 20th century nations to bounce back from the Second World War is also not evidence that ancient or pre-modern societies would have been able to do the same. Medicine and food production were vastly superior in the 1940s and 1950s than they were even a hundred years before.
Technological and scientific advances will explain most of your concerns. Sewage systems, the ability to acquire and create clean drinking water, and the abundance of nutritious foods are some of the most important factors in my mind. Disease prevention is another obvious factor.
Hope some of this helped. I didn’t cite any sources here since it was all so broad, but there are plenty of places to go for information like this.