In a word, yes. Children born to slaves were legally also slaves.

There are actually interesting and little-known laws that existed concerning slavery in the Islamic world. For example, a slave could be married (to another slave) legally by the permission of his master. There were limitations, however. According to the Maliki school of jurisprudence, a slave could have four legal wives, whereas for the other three Sunni legal schools only two were allowed.

Source: Cristina de la Puente, "Slaves and the Slave Trade, Western Islamic World," in Medieval Islamic Civilization: An Encyclopedia.

Somewhat related question: could a slave be freed if he or she converted to Islam? I understand that Islam has prohibitions against enslaving a fellow Muslim, but it seems to me that this would make slavery a very risky business. Since slaves were expensive, a slave owner would stand to lose a substantial amount of money if the slave he just bought converted to Islam and had to be freed. If the slave was to be freed, was he required to pay back his cost?

I know that Christianity had a similar proscription against enslaving fellow Christians, and this had to be specifically legislated against to permit the large slave economies of the south. Various states passed a series of laws that redefined slavery to make it dependent on race rather than religion, and black people could be enslaved regardless of whether they were Christians.

Is there some sort of parallel to this in Islam?

Slaves could be bought & sold. Slaves could conduct business with permission from their masters. Or, 'earn' their freedom by working outside and paying their master whatever amount he demands. If a master has a child from his slave girl, he cannot ever sell the slave girl.