Has it?

Without violating the 20 year rule, the Southern US is definitely not one of the wealthier parts of the country. Many States receive subsidies from the Federal government, which redistributes wealth from states on the Pacific and Atlantic coasts. Those areas which are still agricultural have seen land holdings become larger and larger under fewer and fewer owners. Agricultural production is also heavily subsidized in these territories, particularly corn production.

In the case of the US (cotton), Cuba (sugar/coffee), and Brazil (sugar/coffee/cotton), the prices of the crops that produced so much wealth collapsed by the mid-20th century and commodity prices have continued to decline.

In the 17th and 18th centuries, sugar was a very valuable product which was sold at prices which, compared to today, were outrageous. This meant enormous profits. When Haiti dropped out of the sugar, cotton, and coffee markets this meant that a huge slice of the market was opened up and that prices soared.

By the early 20th century, however, you start running into crises of overproduction in basically all agricultural goods. Production finally outpaced demand, with new territories (including areas in Africa and Asia) joining the list of large scale producers all the time, all leading to major economic crises in the territories which depended on them.

The Southern USA had already been suffering from overproduction crises before slavery was abolished in the 1860s. Sugar overproduction became critical after WWI, with declining prices in the years before 1914.

I'm less familiar with Brazil, but the crisis in Cuba forced the country to diversify its wealth, seek protection under a quota system (Cuba was guaranteed a quota of the domestic US sugar market), and look to tourism for continued growth (especially post-WWII). Sugar would rise and fall in price, but aside from a temporary boost during WWII it remained pretty low compared to 19th century norms.

The major difference between the Southern US, Cuba, and Brazil here is that only the first was part of a major economic powerhouse which was willing (and able) to heavily subsidize its agrarian and less economically developed areas. Even so, the more prosperous parts of the USA are on the coasts and, along the Atlantic, mostly those same States which had been part of the Union and had not been dependent on plantation slavery. Another key factor here is the deindustrialization of the 1980s which put a hurting on places like Detroit, Baltimore, and other locations where a large number of workers had worked in factories or at least economically depended on those who did.

Also, keep in mind that Cuba was a colonial possession of Spain throughout the 19th century, and that Spain brutally extracted wealth through high taxes and tariffs, along with corruption, which all went straight back to Spain. Spanish production was also given preferential treatment over foreign manufactures, to stimulate Spanish industry. Brazil was technically an independent country, and I'm far less familiar with it, but like Portugal it was in practice an economic dependency of Great Britain. The Southern USA was reintegrated into the Union and was not turned into a colonial possession of the North.

Sources:

Sven Beckert: Empire of Cotton.

Luis Perez Jr.: Cuba Between Reform and Revolution.

The Cambridge Economic History of the United States, Vol. 2: The Long Nineteenth Century.

The Cambridge Economic History of the United States, Vol. 3: The Twentieth Century.