There is certainly evidence of anti-slavery actions in the pre-war South, so to a degree you're in the right direction. In the first third of the nineteenth century manumission statistics shoot up precipitously, by some estimates rising a full ten percent in comparison to eighteenth century numbers. Beyond legal manumission, the underground railroad would have been impossible to operate without the cooperation of sympathetic Southern whites.

However these incidents are concentrated in the border region - Virginia, Delaware, and Maryland. The deeper South you go, the rarer these cases show. Furthermore in both cases (manumission and underground anti-slavery actions), the white agents at play would have generally been educated. The likes of quakers, people who for some religious or moral reason found fault with the slave system, and used their clout or the positions they held in their community to agitate against it.

For the average working Southerner we have less to go off of, but it's important to note some statistics here too. It's estimated that as much as seventy percent of the white Southern adult male population served in the Confederate military - in the afore mentioned Virginia, it's believed that number is closer to ninety percent. Contrast this with Union forces, where only about thirty-five percent of men served during the war. So from these numbers we can surmise that the vast majority of Southerners were at minimum tolerant enough of slavery to fight in its defense.

So why was that? Your question, I suspect, stems from an assumption that working class Southerners would have resented slaves for taking jobs they could have had, which is a good question to ask. In the South I am certain there would have been individuals that resented slaves for this, but the reality is we lack any real evidence of this in widespread numbers. The only places we see vocal, and often organized, opposition to slavey in the South are border regions - areas where slavery is only found in specific pockets. West Virginia, Maryland, Delaware, Missouri, and Kentucky all opted to stay in the Union. Their reasons are complicated and not really what your question asks, but let's take West Virginia as an example to illustrate a point I promise I am getting to.

West Virginia's double secession is a curious quirk of American history, but let's look at why it chose to secede from Virginia proper and rejoin the Union. Now it's important to note here WV was already experiencing invasion by Union forces at the time of its secession, so there was outside pressure to rejoin the Union. But in addition to that, slavery was just not the economic driving force of the soon-to-be state. Only about seven percent of West Virginia's population was enslaved, which is a blip next to Virginia proper's thirty percent. Some slave plantations existed in West Virginia, but the majority of slave owners only held a handful of personal slaves. The reality was in WV slavery was not the driving force of their economy, and so West Virginians were far less wedded to the institution (they still did everything they could to hold onto it when they declared statehood, of course).

Now comes the point I wanted to make - those seventy percent of Southerners, unlike West Virginia, had a vested interest in slavery. In the slave holding South slavery was the driving force of the economy. The reason a working class Southerner was willing to defend it, even though he himself held no slaves, is because he recognized that his trade still depended on slavery. They firmly believed that the end of slavery was the end of the Southern economy, and that would bring with it cataclysm for every Southern white, regardless of their direct financial stakes in the institution.

Every Southerner had personal reasons to fight - honor, loyalty, fear for their families - but ultimately they would not have waged the war if they truly felt opposed to slavery. Slavery was an institutional part of Southern life - it was so embedded in the South it's all-but-impossible to discuss the region without addressing it. The simple reality is most Southerns could not visualize a South without slaves, so opposition to slavery would have been seen as opposition to the Southern way of life itself.

For more reading on the subject of the South leading up to the war, I encourage you to check out Aaron Sheehan-Dean's Why Confederates Fought: Family & Nation in Civil War Virginia. He goes into far more depth into this very subject, discussing the motivations that drove working Southerners to fight in the war. Most of my argument is shamelessly lifted from it.

EDIT: as a small note, this is also an area of controversy in Civil War studies. There are definitely those that would argue class divisions in the South was a more serious issue for the confederacy, so as always take this into consideration while digesting my answer. This is what I generally agree with, but others may have different readings.