Several nations considered it, actually. The Union almost went to war with Great Britain in 1861 over the Trent Affair. The Union illegally captured two Confederate diplomats aboard a British ship. War was only averted when the Union released the two diplomats. At the time, France stated that they would support Great Britain if it came to war, but the British distrusted them and believed that the French wanted the British involved in another war to weaken their global power. And that gets at the crux of the matter in a nutshell, while it might have been beneficial for some European nations to invade the US, there was also the chance that it would harm them, which they did not want to risk. Moreover, the European nations were watching each other closely and did not want anyone to gain an advantage over the others.

Nevertheless, there was a strong belief that a divided United States would be better for the European countries than a single nation. Many European conservatives, especially among the nobility, disliked democracy and hoped that the Civil War would demonstrate the ultimate failure of the democratic ideal. To that end, there was a strong sense that they should support the Confederacy, however, the fact that the South represented slavery largely kept that from happening, as slavery was falling out of vogue. Lincoln and his administration were aware of this and created some of the first formal US diplomatic corps to try and get/keep the European people on the side of the Union. To that end they employed newspaper writers, artists, etc. to support the Union cause. Karl Marx actually wrote several newspaper articles about the Civil War and was clearly on the side of the Union. This might seem odd, but if we consider that capitalism was going to lead to communism, then in Marx's eyes the South had not even got that far and was stuck in a feudal stage.

Mediating the conflict was also a possibility, one that the British and French pursued, since they wanted a divided US. The problem here, however, was that neither side could think of a compromise that both the Union and the Confederacy would accept. They tried for some years, but eventually realized that the Civil War had to go all the way before either side would submit.

So, an armed intervention was unlikely, but what is interesting is that the European powers did try to use the distraction of the Civil War to their advantage elsewhere. France, Spain, and British ships went to Mexico, ostensibly to collect debts that Mexico had suspended. The French attempted to set up a monarchy there that would be congenial to French interests. They tried their best and put an Austrian arch-duke, Maximilian I, on the throne for a little while, but as the Civil War was winding down, the US made it clear that it was not going to sit by and let this happen. That, combined with fierce guerrilla fighting in Mexico, forced the French to withdraw and Maximilian was executed in 1867.

Sources: Jones, Abraham Lincoln and a New Birth of Freedom, Jones, Blue and Gray Diplomacy, Doyle, The Cause of All Nations

Not to answer your question with a question, but you have to ask who would want to invade the United States at the time and why? You also must ask who would have been in a position to even consider an invasion of the United States.

Mexico certainly after their defeat in the Mexican-American war, ending over a dozen years previous to the ACW, might have wished to avenge their defeat an recover annex territory. So they might have had the motive. The bordered the United States in the south so they had the position to invade, but did they have the means? Well Mexico had gone through a civil war of their own from 1857 to 1860 and would in 1863 be invaded by France (Who did take advantage of the USA's preoccupation with the civil war, not by invading but by installing a puppet emperor to the throne of Mexico). So mexico simply did not have the means to invade the US at the time.

The only other land border that the United States had was with British controlled Canada. There was some concerns that Britain might have sided with the Confederacy, and events like the Trent affair (a US ship had intercepted a British Mail ship and seized two Confederate diplomats) raised some tension between the US and Great Britain. War between them was a remote possibility the USA even with a war going on and having to feed it's own armies still shipped a massive amount of grain to Britain and France and with poor harvests in 1861 and 1862 these import were critical. The economic losses that going to war with the US would take on Britain were enough to deter serious intervention.

Aside from Britain and Mexico no Other nation was really in a position to invade. Sailing an army large enough to serious contest the power of the US across the Atlantic without having any friendly beachhead (unless the CSA provides you one,) would have been an enormously costly adventure with great risk that no nation would consider unless they felt they had something to gain for it.