I just finished reading Steve Hahn's A Nation Under Our Feet, which deals with this subject. It's an excellent read, and might be available online through some academic libraries (my school had it).

The unstable nature of slave life led enslaved people to construct a new definition of family. One's kin network included all slaves on the plantation (and, earlier, slaves transported on the same ship to the Americas). These kin relations persisted even after slaves were sold to different plantations, particularly if those plantations were nearby (and they often were). This meant that slaves had deep personal ties to their particular part of the South completely independent of their relationships with their masters. These kin networks formed the framework for later black social organization - churches, Union Leagues, militias, etc. It would be difficult to leave all of that behind to move to an area that was not necessarily any less racist (though admittedly less practiced at keeping a large population subjected).

Also consider what skills the freedmen had, and what opportunities were available to them. Devoid of any kind of education, freedmen had to go with what they knew, and what they knew was cotton. For years, there were hopes and rumors that plantations seized by the army would be broken up and given to ex-slaves, or that some other land redistribution program would give them a permanent stake in the land they had worked for decades. Though this got no closer than the sharecropping system, it still let blacks carve out a bit of the old plantations. The various Reconstruction authorities, based in the South and non-existent in the North, offered education and limited health care to freed blacks.

Finally, the option or threat of leaving gave freed blacks leverage in labor negotiations. The white population of the South could not by itself keep cotton yields up, and everyone knew it. By threatening to migrate North or to Kansas or Liberia, freedmen could pressure whites into paying better wages or letting them keep more of their crop. It was a weak bargaining position, but it was better than nothing, which is what they would have faced in the North.