It depends on the time and place but generally they were second class citizens with restricted rights. First of all, its worth remembering slavery was a going concern in connecticut until 1804 and in new york until 1827 and new jersey/delaware right up until the civil war (albeit in limited form). In major urban areas like New York City and Cincinnati major race riots took place that destroyed black property, terrorized communities, and killed many individuals. Most states did not allow black men the right to vote and western states, led by Ohio's example, passed black laws that required black settlers to post cash bonds in order to settle, produce freedom papers, did not allow them to testify in courts, and that did not allow them to attend public schools.

Some communities within these states were more accepting of black settlers and they provided them with more opportunities both socially and economically, but even in the abolitionist haven of New Bedford, MA, Frederick Douglass faced workplace discrimination and had to deal with segregated facilities as he traveled as an abolitionist lecturer. So...here's a quote from the Indiana True Democrat from 1851 that gives you a good idea of what people felt: "[if the south freed its 3,000,000 slaves] and they should begin to arrive freely in the state of New York, e should expect that the free soilers and abolitionists would be amongst the first to oppose them. Seward, Weed, Greeley and Co would probably lead the van [for[ the fact is that in no part of the country is a large, free colored population wanted." (Quoted from Berwanger, The Frontier Against Slavery, pp. 47-48) So..I'd say..pretty awful.

sources: Eugene Berwanger, Frontier Against Slavery Stephen Middleton, The Black Laws