I'm sure there will be better answers forthcoming, but I'd like to through my two cents in. I think there were two primary reasons this didn't happen: geography and capability.
For geography, there are several important factors that make the United State difficult to attack. The first is the presence of large oceans on either side. Any army would have to cross the waters, and given the size and strength of the U.S. Navy at the end of the Civil War, means that only the British or possibly the French could have done it. The British would have the easiest time of of it, since they could simply put their forces in their own North American territories.
Other geographic factors include the size of the country. It was far larger than any other major power at the time (excepting Russia). So, even if a great power could get their army to the country, it would take quite a while to conquer it.
As for capability, war done strange things. Sometimes countries emerge from wars stronger. Other times they are weakened by it. The Roman Republic fought a decade and a half war with Carthage in the third century BC that saw tens of thousands of their soldiers die. However, afterwards, they had a superb veteran army that helped them conquer much of the Mediterrannean.
On the flip side, both the Eastern Roman and Sassanid Persian empires fought a war in the sixth century AD that devistated them. They were so exhausted that, when Muslim forces under the Caliphate arrived, both empires could not hold them back for years.
At the end of the American Civil War, the South was pretty devistated. Sherman's march to Atlanta, the destruction of forces along the Mississippi, and other things lead to a region that would not fully recover for decades. However, the North was a pretty lean industrial machine. They had large debts, but their factories had been up and running for years, refining processes and increasing output. So any invading power would still have to deal with a nation that had a strong industrial base (i can think of maybe two or three powers that would have exceeded the industrial capacity of even half the United States in the late 1860's).
The other advantage the United States had was the veterancy of the army. By the end of the war, the Union Army (including the Army of the Potomac) was a fierce fighting force that could be relied upon to prosecute a war. Other powers would be figthing a "neer-peer" so would have to act accordingly.
This didn't mean the United States was undefeatable. An invading army could almost garentee that the South would be on their side if they promised them independence. Plus, while the North was tough, it was not untakable. And, the industrial centers were just that, centers. If you took certain small regions scattered throughout the North, you could seriously damage the United State's industrial capabilities.
Hope that contributes.
Some further reading:
(Union Army)[https://www.mtholyoke.edu/~ewdow/Politics%20116/aftermath-6.html]
(Economics)[http://www.historycentral.com/CivilWar/AMERICA/Economics.html]