I learned that one of the largest slave owners in South Carolina was freed slave William Ellison, a black man. Being a wealthy slave owner in the American south but also a black man is an unusual thing to think of today, but do we know how William was perceived by other Americans during his time? How did American slaves view a black man becoming a slave owner? Did white slave owners respect him at all? Or did they see William as out of his place?
I'm afraid I don't know enough about other peoples' views of William Ellison to give a thorough answer to this question, but I want to raise a point of clarification in your question, given the fraught, politicized nature of discussing black slaveholders in America.
Ellison was unusually successful free black man in the South Carolina lowcountry, even more unusual for being a genuine planter (a slaveholder who owned more than twenty people). He owned 53 enslaved workers, more enslaved people than any other free black person in the state.
However, he was in no way one of South Carolina's largest slaveholders. This is the point I wanted to raise. His holdings, though quite large by any standard, were dwarfed by the true large planters. Wade Hampton II, who had properties in multiple states, owned more than 300 people. James Henry Hammond also had more than 300. In 1850, Joshua Ward owned more than 1,000.
This is important to bear in mind when you consider how others perceived Ellison. He was one of fewer than 200 black slave owners in South Carolina (out of a total of approximately 700,000 black people, approximately 400,000 of whom were enslaved, so you are correct that it was very unusual). While he was a planter, and would have enjoyed the wealth that came from successful plantation ownership and cotton cultivation done by enslaved workers, he was far from one of the largest slaveholders in the State.