Yes he would be if the owner of the slave wished it to be so. It was considered to be trespassing against the property of the owner. This is how the law codified that it was fine for the owner to rape he slave as it is impossible to trespass against your own property. The only exception is that a slave could kill a white man in self defence. This can be seen clearly in the case of Celia a slave in the 1840s who was tried for the murder of her master. She claimed self defence and that he had raped her for 5 years. It did not work in this case but the very fact that it was used as a defence meant that it could work. To read more about that case and the topic in general I would recommend the following books

Gordon-Reed, Annette, ed. RACE ON TRIAL: LAW AND JUSTICE IN AMERICAN HISTORY (Oxford University Press, 2002).

McLaurin, Melton A. CELIA, A SLAVE: A TRUE STORY (University of Georgia Press, 1991).