This topic has become somewhat mythological and I think its significance has become exaggerated. Malcolm X famously used this dichotomy as a metaphor for a modern division in the black community:
And during slavery you had two Negroes. You had the house Negro and the field Negro....The house Negro usually lived close to his master. He dressed like his master. He wore his master's second-hand clothes. He ate food that his master left on the table....But then you had another Negro out in the field. The house Negro was in the minority. The masses--the field Negroes were the masses....So now you have a twentieth-century-type of house Negro. A twentieth-century Uncle Tom. He's just as much an Uncle Tom today as Uncle Tom was 100 and 200 years ago.
Meanwhile, you also had historians like Eugene Genovese in the '60s who tried to downplay the historical divisions between house slaves and field slaves in order to promote black unity. So it's always been a politically touchy subject.
The first thing to note is that in 18th and 19th-century America, there were many households in which there were so few slaves that the workload was not neatly differentiated. In 1850, the average slaveholder in the Lower South owned twelve slaves; in the Upper South, five. The "Gone With The Wind"-style plantation with a large and specialized workforce was important but atypical. For masters, it was more important that the labor force participation rate stay high throughout the day and year, and this worked against strict divisions. During the harvest, the household slaves would be out in the fields. Children were better suited to domestic work, and so would be working around the home until they got old enough to go into the field.
About 1% of enslaved men and 5% of enslaved women worked in households. Again, this is because only the great planters could afford such luxuries. Men within households were personal manservants. Domestic women did a wider variety of activities including cooking, childcare, dairying, raising poultry, marketing, manufacture, and repair.
To complicate things further, there were those skilled workers on the plantation who fit in neither category. These included drivers and foremen, artisans, and watermen.
House servants probably adopted a separate identity. They had typically been "bred from infancy" for this type of work. They had more mobility than fieldhands, ate better food, had more interesting lives, and were more likely to be literate. Former slave Henry Bibb wrote, "The distinctions among slaves is as marked, as the classes of society are in any aristocratic community." An interesting discussion of the self-consciousness of these slave elites is Charlene M. Boyer Lewis, Ladies and Gentlemen on Display: Planter Society at the Virginia Springs, 1790-1860.
At the same time, housework was no walk in the park, as it involved all the brutalities which fieldworkers faced. Field hands would know intimately that this was the case, and that the privileges of domestic service were not especially sweet. Jacqueline Jones in Labor of Love, Labor of Sorrow argues that many enslaved women preferred working in the fields to working in the house, primarily because there was less surveillance. Domestic servants had to be on call 24/7. There are many cases of domestic workers "betraying" their masters by murdering them, poisoning them, absconding, etc.
The house slave/field slave division should not be given undue significance when it comes to the internal dynamics of the slave community. We also need to consider differences between Africans and creoles, ethnic differences, gender roles, regional origins, skin color, and the particularities of each conflict. Masters used every angle they could to divide the slave community. Those who reported the wrongdoings of other slaves would be rewarded. So if there was a dispute between two enslaved men or women, there could be many factors in play.
I don't know if this is a satisfactory answer but it's the best I could do.