Well, first off you are working from some incorrect assumptions. The average African-American today is slightly shorter than the average non-Hispanic White American, though taller than other groups like Hispanic or Asian. I can't comment on musculature but I would imagine they are similar.

Secondly, people didn't keep very good records of slave births other than that they happened and if the baby died and when. They also didn't have a great understanding of the causes of birth defects or nutrition. It wasn't a big concern for the owners and most slaves could not write. Slaves often had infantile death rates double or triple that of white contemporaries, so I think it is safe to assume that not only did they have more birth problems, but poorer health in general. Most slaves were also shorter than whites as well, though they were strong and tough due to the physical labor. They would not have been any stronger than a well fed white laborer or farmer though. Most slaves in antebellum America grew their own crops in small fields when able, and would have given extra food and nutrition to pregnant and breastfeeding women. They were not wholly dependent on their masters for food.

As for African-Americans appearing as healthy as everyone else today, it generally only takes about three generations for the effects of extreme poverty to be corrected. Smaller, underfed people, have smaller hips to give birth to smaller children. The effect is slowly reversed with better fed people having bigger babies over a few generations. This has been observed in China and Korea, for example. Since five to seven generations have passed since slavery was abolished, it follows that most of the physical legacy from it is gone at least.