I think there are two main factors that will answer your question.

First, we should look at the population size of the Confederacy—including slaves—and the amount of casualties they suffered during the war. It’s estimated the south had 260,000 soldiers killed in the Civil War. The population of the Confederacy was around 9.1 million in 1860. In contrast the Union had a population of around 22 million citizens and lost 360,000. Doing some rudimentary math, the Union lost 1.6 percent of its population; the South lost 2.8 percent when you include slaves, and 4.8 percent when you exclude them. This latter number is more accurate since the slaves weren’t officially allowed to serve in the Confederate army as soldiers until near the end of the war in 1865, and in a limited capacity at that.

The second factor that goes towards the “brain-drain” is the existing education levels of the states. A good measure of this is how many people were able to read and write. James McPherson speaks on this directly when he notes, “New England led the world in educational facilities and literacy at the midcentury. More than 95 percent of its adults could read and write; three-fourths of the children aged five to nineteen were enrolled in school” he concludes by saying, “The rest of the North was not far behind.” Meanwhile, in the South, only about eighty percent of its white population were literate and one third of the white children were enrolled in school for an average of three months per year. Much of this can be attributed to the South's dependance on agriculture which was declining in the North.

But the most telling statistic is when you look at the slaves. They comprised 3.5 million citizens and only one-tenth were able to read or write. Upon being set free, it took many years before their reading and writing levels, as a whole, would rise.

So the answer to your question rests in a combination of the South losing a higher portion of their citizens during the war and the fact that once the war ended, and the slaves were freed, only a small portion of them knew how to read and write. You couple this last fact with the lower rates of literacy for the white Southern population, and you see the seeds of "brain drain" were planted throughout the history of the South and only became more apparent after the war.

J. McPherson. “Battle Cry of Freedom.” Pgs. 19-20 and 854.

"Population of the United States in 1860," Eighth Census of the United States, Bureau of the Census Library, Sup. Joseph C. G. Kennedy, Bureau of the Census, http://www2.census.gov/prod2/decennial/documents/1860a-15.pdf