I wouldn't say it's a particularly American thing. I'd be surprised if it was anything other than 'south' being synonymous with 'down', like on a line graph when the line goes down/south it's generally a bad sign. I'd say it's as simple as that. I've certainly never seen any research to suggest it has anything to do with the American south or geographical south in general.

This site links the term with the standard way to depict the cardinal directions, with south, depicted as going down. "Down" = lower value and going bad. I suspect there may be a pre-literate version of this. People in the Northern Hemisphere tend to think of the south as down (without seeing how the directions are identified on a map), and "going down" would be easily linked to "going to hell." I see no evidence of a Civil War link.