It seems to me that the region surrounding a large river that already featured a thriving port (New Orleans) would have been a prime target for industrialization. While New Orleans remains a major trade hub, move a little to the north and you'll find a region that is relatively underdeveloped and features some of the poorest parts of the US. How did this state of affairs come to pass?
You're actually understating the importance of New Orleans in the early south. It was basically the only major city in the Confederacy. It was the largest city in the south and larger than the next four combined. The capture of New Orleans was so important because the South could credibly threaten major midwestern economies by cutting off the port. On top of that, the Lower Mississipi has incredibly rich farmland. So why is a region blessed by a navigable river, a major port, and incredibly rich soil so poor? The region even has abundant hydraulic resources for early industrialization.
Well, prior to the Civil War, Mississippi and Lousiana were some of the richest states in the Union. This was incredible at the time since the states were underpopulated and very new. Imagine if two of the richest states today had been incorporated in the 1970s and had a population of less than two million people between them.
But it was wealth almost entirely bound up in plantations (and in the one major city, New Orleans). The vast majority of the land was actually empty and the state government mostly encouraged plantations rather than small farms. While these brought in more money, it meant settlement was slower and the population was small. There was also a vast divide in wealth between poor whites and those who owned plantations. So while wealthy, all that wealth was invested around slavery and the plantation economy and there were serious structural vulnerabilities in the economy.
The Civil War hit the region like an avalanche. During secession, several parts of the midwest actually threatened to side with the Confederacy. While they had little sympathy for slavery, they were agricultural economies (and thus had similar economic interests), had similar feelings of domination by the populous northeast, and relied on southern ports to sell their goods. Some of Lincoln's policies can be seen as attempts to appease the midwest, as Lincoln was, in general, willing to compromise with anyone who was pro-Union and anti-slavery. One of the key ones was the taking of the Mississippi, and thus keeping the port open and the midwestern economy from collapsing. New Orleans couldn't be blockaded like other southern ports: it needed to be opened.
The Union hit New Orleans by land and sea. The Union succeeded in capturing the city in 1862, early in the war. The actual battle itself was not particularly destructive. New Orleans had significant Union sympathies and most of the resistance was from defenses around it. But between one of the most thorough slave economies in the north of the states and the pro-Union, occupied south (and with a lot of land to hide in) a devastating guerilla war broke out. The plantation economy collapsed and much the countryside emptied, devastating the agricultural economy. By the end of the war, the region's economy had shrunk by 60%.
After the Civil War, something like 90% of rural land was still unclaimed. Poor whites and freedmen settled the region with the encouragement of the Republican government (with about a 1:2 ratio in favor of freedmen). Many plantations still remained too. The region began to recover under Reconstruction, though nothing like it had been, and even began to industrialize somewhat.
Then the Redemption happened. The Democrats came into power and gained total control, even rewriting the constitution of Mississippi. The Democrats were hostile to business, corporate industrialism, and the freedman and poor white small farms. They firmly put the region on an agriculturalist, big-farm economic course and were hostile to other modes of production. There was also a campaign of terror, one of the earliest and most thorough, which destroyed productive small farms and drove a lot of labor out of the region. A region that was already underpopulated. Some estimates say as much as 10% of the population fled. What infrastructure spending there was went into levies, land reclamation, and other agricultural projects.
With the exception of the terror, most of this was defensible, if short-sighted, economic policy. Cotton was still a productive endeavor and they weren't just invested in cotton. They produced numerous other agricultural goods, like rice or other food crops. The levies were necessary to prevent flooding, which can still be seen today in disasters like Katrina. The state did gain some wealth and there wasn't an immediate economic collapse. It's easy to look back in hindsight and see the mistakes since we know the collapse is coming. But they didn't know that and, if factories hadn't taken off, we might be making fun of Massachusetts for neglecting its farming hinterland. (Though, in counterpoint, it shouldn't be ignored that ideological hostility to the poor, minorities, and corporations hurt the economy for little benefit.)
But agricultural prices did collapse. By the time of the Progressive era, the state's finances were simultaneously bad while sustaining one of the lowest levels of infrastructure in the nation. There was a great deal of success in battling this, particularly among whites, but open racism meant little education or healthcare for the African Americans who were about half the population. Indeed, the most progressive and successful Democratic politician in the area was a member of the KKK. Nevertheless, the economy of the area began to grow and there has been steady improvement ever since.
But that improvement has been slow, something like half the general US growth rate. While the region had always been relatively poor, it has been getting richer relative to itself. While each generation of Mississippi has done better than the last since about 1900. They just started out poorer and have grown more slowly. The economy continues to suffer from lack of population, lack of infrastructure, rurality, lack of education and healthcare, deep wealth gaps in agriculture, low agricultural prices, and people leaving the state. It has a great deal of endemic, generational poverty, a high unemployment rate, and a low labor force participation rate. Its tax base still relies disproportionately on the wealthy and is very low. It also continues to have an economy that focuses on things like agriculture and fishing, with more than half the labor force participating in such activities.
But this has always been the state of affairs to some extent. Mississippi didn't become poor. Mississippi has always been poor, except for a fifty year period when the slave system made its wealthy elite ridiculously wealthy. (And even then, that was of dubious benefit to the other classes.)
Sources