The differentiation between indentured servitude and "servitude for life" was first acknowledged in the US by a 1661 law by the Colony of Virginia. Most of the early colonists in North America were indentured servants that usually submitted themselves to a servitude working agreement in exchange for the funding for the crossing to America. Indentured servitude was established with a time limit and that time limit was (usually) strictly enforced under British common law. However, upon the enactment of that Virginia Law, people of black skin color were exempted of such protection and could face life servitude.

In December 1662, the Virginia assembly enacted another law (Act XII) that explicitly determined:

Whereas some doubts have arisen whether children got by any Englishman upon a Negro woman should be slave or free*, be it therefore enacted and declared by this present Grand Assembly, that* all children born in this country shall be held bond or free only according to the condition of the mother*; and that if any Christian shall commit fornication with a Negro man or woman, he or she so offending shall pay double the fines imposed by the former act.*

As result, the status of children born to a woman of black skin color and a white man would depend solely on the status of their mother.

It is interesting to observe that such a law was based in the principle of "partus sequitur ventrem", a legal dictum of Roman origins based on which the social status of the child should reflect on that of the mother rather than that of the father. Such a concept went against the whole British legal primogeniture tradition, which was settled around the figure of the father. Under British common law, children always claimed lineage through their father, even if they were born out of the wedlock. That happened because the colonies were left free to define any standards concerning slavery, as slavery was not a condition recognized under British common law.

Hence, Virginia was the first of all British colonies to create a system that referred to the mother to define the status of a newborn. And even within the Virginia law that was also an exception, applying only to mothers of black skin color under a condition of servitude to life, as the same law didn't apply if a white female indentured servant gave birth to a child fathered by her master. White female indentured servants were submitted to specific legislation that didn't apply to black female slaves.

Other slave colonies (and later, after independence, Slavery-allowing slave states) followed suit and enacted similar laws. For instance, The 1740 Slave Code of South Carolina determined, in its first paragraph, that Slavery was hereditary and perpetual and the children would also follow the condition of the mother:

And be it enacted, by the honorable William Bull, Esquire, Lieutenant Governor and Commander-in-chief, by and with the advice and consent of his Majesty's honorable Council, and the Commons House of Assembly of this Province, and by the authority of the same, that all Negroes and Indians, (free Indians in amity with this government, and degrees, mulattoes, and mustizoes, who are now free, excepted,) mulattoes or mustizoes who now are, or shall hereafter be, in this Province, and all their issue and offspring, born or to be born, shall be, and they are hereby declared to be, and remain forever hereafter, absolute slaves, and shall follow the condition of the mother*, and shall be deemed, held, taken, reputed and adjudged in law, to be chattels personal, in the hands of their owners and possessors, and their executors, administrators, and assigns, to all intents, constructions and purposes whatsoever;*

So, to answer your question, by law, children born to black slave mothers even if fathered by any white men, including the mother's owner, would be slaves for life, and property of the mother's owner.

As you can imagine, life was awful for those mixed race children, as the following excerpt of "White Slaves”, which was one of several pieces on the South printed in "Youth’s Companion" as the Civil War ended in 1865::

We offer our young friends to-day, the portraits of four youths, who were formerly slaves*, but who have been made free by the successes of our army and navy in Louisiana. They lived in New Orleans, and are now probably studying in the schools which Gen. Banks established in that city for the education of the children of freedmen.*

One of them, the tall figure in the background, is a full-blooded negro. His name is Robert Whitehead. The boy, who is called Isaac White, is of light complexion. The two girls are perfectly white, and are named Rebecca Huger and Rosina Downs.

Let our friends not be surprised that these girls are white. They are in fact as light colored, and we dare say, as good looking as most of our young readers of their sex. And so it often is at the South. Large numbers of the slaves are white. This indicates one of the evils, and the great wickedness of slavery. The white slave children are not unfrequently the children of their masters. Often a number of children will grow up together in the same family. They play together as companions. They are all as fair to look upon as you or I, and in the sight of God there is no reason why there should be any difference between one and another. They are brothers and sisters, for they have the same father. Yet a part of them are slaves. Why? Because their mother was a slave. The others are free born, for their mother was the wife of their father. The poor white children of the slave mother are sold like brutes to the highest bidder, by their worse than brute father, while their free born brothers and sisters*, who are not whiter than they in complexion, or purer in heart,* inherit the father’s wealth, and enjoy the blessings of that freedom which is the choicest earthly gift from God to man. Thus slavery degrades and makes fiendish the dearest relations and the purest instincts of humanity.

However, if that was true for most of those mixed-race children, in fewer occasions, some of their fathers exhibited some "affection" toward their children. As result, there were some instances of fathers emancipating their slave offspring and sending them north to receive an education and to enjoy life as free persons. This would apply mostly to boys, as was not common for girls in the 1860s to be sent away from home to fend for themselves.

But, for other children that were not so fortunate, and didn't receive any affection from their slave-owner fathers, life would not be so easy though as they would end up being simply sold to a life of slavery under a different non-family master, out of sight from their father and, especially, his white wife.

As a population of light-skinned slaves developed, most of them became preferred to fill the role of house slaves, which, for that reason, tended to be lighter-skinned than field slaves, and would command higher prices when sold, especially whiter girls that were pretty and that would command a high premium price, for pretty much obvious reasons.