Edit: I know it is tacky to acknowledge the gilding of your own post and all, but while first off "Thanks!", I have a bazillion years of Reddit Premium already and don't need more, so if anyone else is feeling that impulse, please instead consider making a donation of that amount to a charity that helps sexual assault survivors. The below might be history but it also isn't in too many ways. Thanks. (Edit II: I'd recommend RAINN or Children of the Night)

The sum is that you would likely turn a blind eye and pretend that your husband wasn't routinely raping the enslaved women that he owned, and deny to yourself the strong resemblance that many of the lighter skinned black persons on the plantation might have to your spouse. It would be known, of course. Gossip existed, but the diarist Mary Chestnut famously summarized it that "any lady is ready to tell you who is the father of all the mulatto children in everybody's household but her own."

In more detail, of course, I have written about the broader topic before, which further touches on several aspects of your question, as well as other issues of the treatment of enslaved persons, which I'll repost here:

Before we continue, a word on definitions. Rape is a term that can be applied to essentially any sexual relationship between an enslaved person and their master. The practical forms which master-slave sexual relations took ran the gamut from brutal and forced submission to 'real' relationships, but it cannot be separated from the framework in which they occurred, namely the actual legal ownership of the enslaved woman and rights to her body. No matter how willfully a slave-woman (or man) acquiesced to a sexual relationship, their consent within that framework cannot be entirely separated from the fact that the consent was not required, and was given with that understanding. It is a balancing act, really, as we both don't want to overlook the systemic framework in which the sexual relationship occured, but as the same time in looking at it broadly, we musn't deny the agency of some slave women who, within that framework, nevertheless did at times have some choice, however limited. Put another way, if asked "Was it always rape?" the answer is "Yes, but no, but yes": The power-dynamic intrinsically places it within that framework; but we shouldn't deny the women agency; but we then shouldn't overcorrect and let that agency trump the fact that they had no choice to be within the system which gave them the limited choices they did have. On a macro level, yes, it was always rape, but that shouldn't stop us from seeking to understand the intricacies and realities of the actual lived experiences of those enslaved women (and men).

As you bring up Jefferson-Hemings, this is worth mentioning as it is a good example of the complicated nature that these relationships could reach, although it is also something of an outlier. Annette Gordon-Reed's work on Sally and the Hemings family in "The Hemingses of Monticello" is an excellent work that spends a good deal of time exploring the relationship, and more importantly exploring Sally's side.

Sally though, had far more opportunity than almost any other enslaved woman who found herself the object of the master's sexual advances. She was rather unique in being in France with the Jefferson family at the beginning of the relationship, which meant that technically under French law, she was free and could have certainly succeeded in a petition for it to the French authorities. Although she herself left nothing on the subject, but their son Madison, long after booth his parent's deaths, related what the understanding had been under which she agreed to return to Virginia:

But during that time my mother became Mr. Jefferson’s concubine, and when he was called back home she was enciente by him. He desired to bring my mother back to Virginia with him, but she demurred. She was just beginning to understand the French language well, and in France she was free, while if she returned to Virginia she would be reenslaved. So she refused to return with him. To induce her to do so he promised her extraordinary privileges, and made a solemn pledge that her children should be freed at the age of twenty-one years. In consequence of his promises, on which she implicitly relied, she returned with him to Virginia.

So in short, she was in an incredibly unique position to have actual leverage. Gordon-Reed spends a good deal of time exploring the complications of this relationship, and I won't really dwell further on it other than to say that Jefferson mostly kept his word, but the relationship is so unrepresentative of the general circumstances we see in the south sexual relationships between slaver and enslaved, that it really ought not be the focus, even if it can't be avoided simply due to its fame and prominence.

Practically speaking, the extent of enforced, legal protections that a slave woman had against sexual abuse essentially related to the damages that she might sustain if raped by someone else, in which case, of course, the offense was against her owner, not herself. It is of course supremely ironic, that in this situation whether or not the black woman consented had no bearing. The offender had violated the master's property rights, and severe sentences were common. There were some laws concerning 'miscegenation' which in theory could see a white man in legal trouble (but not for the rape part), but their enforcement was never common, and unheard if by the antebellum period. I say all of this because while relationships described may not always be violent, they absolutely must be understood within that context, and I don't want it forgotten with the following. It was a constant threat that slave women lived with over their heads, whether manifested or not. Linda Brent, a slave woman (and pseudonym for the writer Harriet Jacobs), sums up these fears well when describing how she "entered on my fifteenth year—a sad epoch in the life of a slave girl":

there is no shadow of law to protect her from insult, from violence, or even from death; all these are inflicted by fiends who bear the shape of men. [The slave girl is] prematurely knowing in evil things. Soon she will learn to tremble when she hears her master’s footfall. She will be compelled to realize that she is no longer a child. If God has bestowed beauty upon her, it will prove her greatest curse. That which commands admiration in the white woman only hastens the degradation of the female slave.

Now, as to the matter of masters (and younger male family members, and overseers) and their sexual relations with enslaved women in the antebellum South, it was fairly common. There was a decided view of the black woman as being naturally promiscuous and sexual (compared to the belief in white women being chaste and demure) which only helped to encourage the behavior. But although it was a common occurrence, it was definitely not something talked about in polite company, and doubly not around women, although they often knew what was going on - speaking of the sexual relations that the menfolk took, the famed diarist Mary Chesnut wrote of black women that "we live surrounded by prostitutes". It was essentially something that most of white society would just pretend didn't happen, no matter what the evidence, of which it often could be fairly clear, as recalled by one slave:

[Master] used to have some Irishmen on the plantation, and he said these children were theirs, but everybody knew they were his. They were as much like him as himself.

Another example relates a master who accused his childrens' tutor of fathering the biracial child of a female slave on the plantation and dismissed the young man, although many believed the master himself to be the father and simply using it as a 'cover'. No one, of course, would call the man on it though. And the slaves themselves wouldn't dare even acknowledge it among themselves but in secret, as to do so could result in severe punishment.

Jacobs' time with her master, Dr. Flint, was punctuated not only by his advances towards her, but to others women enslaved in his household. At least one black woman bore his child, likely unwillingly, and when her own spouse raised the issue of the lightened skin of their son during an argument within earshot of an overseer, they both suffered for it, being sold off in short order for speaking what was expected to remain quiet. While many followed the strictures laid out by Chestnut, some white mistresses, as in the case of Jacobs', were quietly jealous, but they had few ways to vent their anger, which might simply manifest itself in worse treatment of the women that they suspected to be the object of their husbands' attentions - caring little how wanted those attentions might be. Jacobs, at least, was fortunate enough in that her jealous mistress worked to prevent Dr. Flint from acting on his licentious thoughts, but not out of a sense of moral uprightness, so much as her jealousy. In rare cases, the most moral of women might attempt to convince their husbands to free their literal children, but as Jacobs noted, "bad institution deadens the moral sense, even in white women, to a fearful extent".

As I already noted, it wasn't criminally rape to literally rape your slaves, so the law presented no impediment to a licentious master, and the only real protections were thus unreliable at best, such as Jacobs', who was saved not by grace but by jealousy. The threat of community censure also could provide some protection, but limited at best, since it was generally only "concerning" if a master flaunted the relationship, as opposed to keeping it quiet, and even then, it was no guarantee the community wouldn't willfully turn a blind eye. Bertram Wyatt-Brown sums up the so called 'rules' that were to be followed thusly:

Related question that I hope u/Georgy_K_Zhukov could answer: How did these power dynamics of race and sex shift and evolve during and post-Reconstruction?

We know that there were all kinds of Jim Crow laws, and that the Klan used the threat raped white women to justify their violence towards blacks. But were there many recorded cases of interracial marriages in the South during Reconstruction, and were they more common in certain areas like near New Orleans?

Related question: was this something more common than not during this time period? Like was pretty much every slave owner taking advantage of their slaves in this way?

Your answers were fascinating. So much so I've just ordered the book by Gordon-Reed.

Edit: can't spell.

Follow up question: how did this pertain to Latin America and European colonies in the Caribbean/South America that also had African slaves? The British definitely treated slaves differently than the Spanish and Portuguese did.