One way of answering this question is in terms of consent as we understand it today. Thomas Jefferson literally owned the bodies of all his slaves. It's hard for modern people to conceive of that sort of power relation in any way, but that's what it was. He completely owned them; they had no power of consent over any aspect of their lives. In such a situation, any sexual encounter between master and slave is coercive, and meets the definition of rape. Therefore, Thomas Jefferson raped Sally Hemings, repeatedly and over a number of years. In this view, to talk of different degrees of rape, whether the rape was "violent" or "forced" is abhorrent, and can't be countenanced. I admit, there's a lot of power in this perspective, not least because it foregrounds the immoral criminality of slavery in absolute terms.

Annette Gordon-Reed, in her fascinating book The Hemingses of Monticello, which seeks to pull away from the larger history of slavery to understand one family (Sally Hemings, Thomas Jefferson, and their descendants), takes a somewhat more nuanced perspective. Reed, of course, acknowledges the monstrosity of slavery and its sexual power relations on a world-historical scale, but finds that perspective inadequate for her more targeted, micro-historical approach. Such a perspective, in her view, completely obscures the agency that enslaved women had--or may have had--over their own sexualities. It makes sexuality an entirely male-focused affair, rendering the question entirely "Did so-and-so have sex with a slave?" "Yes? OK, it's rape." It leaves no room for the possibility that Hemings or another enslaved woman might have had a real emotional and physical interest in an owner. Put another way, it renders entirely ideological (masters vs. slaves, in never-ending war), a relationship that surely must have had more nuance. To take the first view I enumerated, ultimately, in Gordon-Reed's view, serves to blot out the real lived experiences of enslaved women. These experiences are very hard for us to understand today, given the sheer paucity of literary and archival sources that directly convey the experiences of enslaved women. So she's hesitant to paint with the broad brush, because ultimately that brush serves to obscure understanding, not enhance it. It can even serve to render enslaved women eternally childlike, which itself was a goal of the ideology of slavery.

You might be interested in reading Gordon-Reed's book to further tease out the nuances of her argument. It's a really great book, though Gordon-Reed is trained as a lawyer and not an historian, and other historians have had various issues with the book's methodology.

There's no evidence that Thomas Jefferson, a refined and charming man who considered himself a statesman, ever held an enslaved woman down at knifepoint and raped her. Given the power relations between him and his slaves, it's doubtful he ever would have had to. So, I believe you are on firm historical ground if you want to claim that Jefferson raped Hemings. But there is at least one decent argument to say that the answer is more like "we don't know, and probably never can."