I can't fully answer your question, but I can help you with some.

First, a black person would most definitely not have to be a slave, at least not per the laws of the U.S. or any State. The idea that black skin de facto made you a slave didn't arise until immediately before the Civil War, and only in Mississippi as far as I know. See Mitchell v. Wells, 37 Miss. 235 (1859) ("A negro, by the laws of this State, is prima facie a slave")

Second, I do not know about the source (i.e. the background) of free blacks throughout the U.S. pre-civil war, but I do know that anyone of the "negro African race" could not become citizens. See the infamous Dred Scott v. Sandford, 60 U.S. 393 (1857). Justice Taney's language is pertinent to your question, because his language makes the assumption that anyone of African descent was a descendant of slaves.

edit: I should note that the Dred Scott decision was 1) not until 1857, leaving a lot of history before that and 2) was met with a lot of backlash among scholars, activists, and some northern states. Dred Scott was specifically asking whether African-Americans could attain Federal citizenship, not State citzenship, which several states allowed, IIRC. My memory of this is from Battle Cry of Freedom by James McPherson.

Follow up question: would groups such as Ethiopians, Somalis, Berbers, and other populations that made up the "Hamitic race" be considered "black" under the US laws of the period or be treated as an Oriental population similar to Arab and Syrian immigrants? Did such popular "racial science" ever intersect and inform US legal definitions of race?