Not an expert in this particular field of 19th century African American history, so feel free to correct me with impunity. But I am going to take a stab at it.

As far as I can tell (and if someone has better information please share), coming from the book Slaves Without Masters: The Free Negro in the Antebellum South once again by Ira Berlin we can say that substantial numbers of the roughly 3.5 million former-enslaved people remained or later returned to their former "masters". I know, a very unsatisfying answer. But let me explain why.

This is a very complex question mostly because what constitutes "working legally" and getting "paid" are very nebulous and subject to change in the chaotic days of Reconstruction. Sharecropping and convict-leasing were perfectly legal ways to hold African Americans in slavery in all but name. In addition, numbers are hard to come by because the US Census did not start recording former enslaved people's names until 1870. Which is only worsened by the fact that each Southern state tackled African Americans a little bit differently. Not only that, as written in the book Slavery by Another Name by Douglas A. Blackmon and Generations of Captivity by Ira Berlin make it clear that just because Slavery was abolished does mean that involuntary labor was gone as well. To the contrary, sharecropping and convict-leasing kept African Americans (and by the 1930s-poor whites) in conditions that were very much similar to chattel slavery. On top of that, many African Americans were coerced to remain either because of Northern apathy, lack of protection, resources, systemic Southern oppression and an overall lack of resources. To top it off, Jim Crow reared its ugly head and systematically pushed African Americans out of politics and kept them separate and very much unequal.

It really irritates me when I go to historical house museums and hear that "that the (former) enslaved chose to remain and work for their ex-masters because they were good masters." No, simply, no. To do so over-simplify this complex issue and ignores the the underlying issues around Reconstruction and post-slavery America.

​

Sources:

Berlin, Ira. Generations of Captivity: A History of African-American Slaves. Harvard University Press 2004.

Berlin, Ira. Slaves Without Masters: The Free Negro in the Antebellum South, Pantheon Books, 1974.

Blackmon, Douglas A. Slavery By Another Name: The Re-Enslavement of Black Americans from the Civil War to World War II. Anchor Books 2008

Downs, Jim. Sick of Freedom: African-American Illness And Suffering During The Civil War And Reconstruction. Oxford University Press, 2012