The fact that he gave white slave owners their land back, allowed for slavery to continue under the guise of tenet farming/share cropping and fought Congress on how to best reunite the nation makes me believe that he's a good underlying cause for why we could never get a strong working middle class of African Americans in the South in this country. Do you believe it was soley him? I mean, I know Lincoln only freed southern slaves as a war tactic and disagreed with giving them full rights but Johnson had a chance to set it up right and he failed, in my opinion.
Would you attribute this problem to another previous president, Johnson, or say that all presidents had some helping with this problem?
There were other failed opportunities later. I think you should look more into racial exclusion in the twentieth century, when the white middle class developed into what we understand today. Richard Rothstein's "The Making of Ferguson" is worth a read. I won't summarize it fully, because you can just read the executive summary. He describes how government at every level ghettoized African American people with policies that were blatantly segregationist. Wealth inequality, wage inequality, and education inequality all reinforce each other.
This is just one sample (fresh in my mind) of racial policy that obstructed black Americans. It's not the full story, and you can certainly look further back. I think it's important to note that something like this is not due to a single failure, but multiple instances where people chose to not fix the problem.