I'm researching my family's history and I've discovered that in the 1860 census my family is listed as living with Isaac, a free black man (age 25). Isaac was born free in North Carolina and moved to East Tennessee (where my family lived) as a young man and lodged with my (white, slave-owning) family. How would my family have viewed him? Would they have thought of him as fundamentally different to their slaves? Two of my family members were about the same age (a 25 year old man and a 21 year old woman), would they have been friends with Isaac or thought of him as lesser to themselves? Isaac also started a relationship with one of the slaves my family owned and had several children with her? How would this relationship have been perceived? Was it normal for lodgers to 'date' slaves?
I realize that you can't know exactly what my ancestors though but I'm trying to understand what the average white family would have thought in this situation. I've come to realize through reading about this that my grasp of race relations in the pre civil war south is sorely lacking.
This older answer deals more broadly with manumission (freeing an enslaved person) but discusses life for them afterwards at several points so should be of interest for you.