Any comparison of growth rates among cities finds a host of underlying reasons, from climate to power sources to local political leadership. But I think the primary reason is that the cities that outgrew the Southeastern cities were diversified cities with extensive manufacturing and regional finance sectors. The cities left behind were primarily ports exporting agricultural commodities—notably cotton and tobacco—whose importance declined.

The migration of workers to the factories built in the rise of industry around 1900, especially the automotive industry a few decades later, played a big part in the population shift.

Also a number of those cities were agricultural. For example, in the mid 1960's, Stax Records and the rest of the music industry supplanted cotton as Memphis' largest moneymaker.

It is worth pointing out that Baltimore was very much a Southern city in the 19th century. Moreover cities in the North were already much larger than the average Southern city by the mid 19th century.