It might be Hinton Rowan Helper of North Carolina. His book, The Impending Crisis of the South: How to Meet It (1857) caused a fair amount of consternation in Southern politics before the Civil War.

His antislavery argument focused on the harm that slavery did to whites, saying that it drove down the wages of free people and discouraged innovation and progress. He compared slaveowners to thieves, saying, "the thief rarely steals from more than one man out of an hundred; the slaveholder defrauds ninety and nine, and the hundredth does not escape him. Again, thieves steal trifles from rich men; slaveholders oppress poor men, and enact laws for the perpetuation of their poverty."

In response to antislavery thinkers, proslavery advocates sometimes pointed toward the individual freedom of whites. If a white man didn't agree with slavery, they said, nobody was forcing him to purchase slaves -- but he should let the slaveowners make their own decisions. In response, Helper wrote:

"[W]hen we, the non-slaveholders, remonstrate against the continuance of such a manifest wrong and inhumanity . . . they fly into a terrible passion, exclaiming . . . 'It's none of your business!' --meaning to say thereby that their slaves do not annoy us, that slavery affects no one except the masters and their chattels personal, and that we should give ourselves no concern about it, whatever! To every man of common sense and honesty of purpose the preposterousness of this assumption is so evident, that any studied attempt to refute it would be a positive insult. Would it be none of our business, if they were to bring the small-pox into the neighborhood, and, with premeditated design, let 'foul contagion spread?' Or, if they were to throw a pound of strychnine into a public spring, would that be none of our business? Were they to turn a pack of mad dogs loose on the community, would we be performing the part of good citizens by closing ourselves within doors for the space of nine days, saying nothing to anybody? Small-pox is a nuisance; strychnine is a nuisance; mad dogs are a nuisance; slavery is a nuisance; slaveholders are a nuisance, and so are slave-breeders; it is our business, nay, it is our imperative duty, to abate nuisances; we propose, therefore, with the exception of strychnine, which is the least of all these nuisances, to exterminate this catalogue from beginning to end."

Despite some passages in the book that refer to the suffering of the slaves, Helper was far from being an advocate of black rights. His concern was for white people, and he was extremely racist.

Sources:

Hinton Helper, The Impending Crisis of the South: How to Meet It (1857), pp. 139-140. The full text of his book is available online as part of the University of North Carolina's project, "Documenting the American South," at https://docsouth.unc.edu/nc/helper/helper.html

Ronnie W. Faulkner, "Hinton Rowan Helper (1829-1909)" at The North Carolina History Project, https://northcarolinahistory.org/encyclopedia/hinton-rowan-helper-1829-1909/