As a layman I have often heard that slavery was responsible for the South being behind the north, that it benefited the few and not the larger economy and so on.

However, I was just reading a book where Francis Fukuyama acknowledges and then dismisses these arguments. The relevant quote:

There has been a prolonged and often bitter debate among historians over the economics of North American slavery. Some have argued, along with a number of slavery critics prior to the Civil War, that the practice was an economically inefficient one that could not compete on equal terms with free labor, and would have died out on its own under free-market conditions. A number of Marxist historians have argued that the Civil War itself was driven not by moral considerations concerning slavery but rather by a competition between free and servile forms of labor. On balance, however, it would seem that right up to the time of the Civil War, slavery-based plantation production was a fully competitive form of economic enterprise and that per capita incomes in the South began to decline relative to those in the North only after the war and the abolition of slavery

I'm curious to see if there's any historical consensus on this topic. Was the per capita income in the South not declining until the end of the war? Was slavery at no risk of "dying out" due to being uncompetitive?

Thanks