This question comes up in this sub rather often, and I have addressed it in previous answers here and here.

To get a full answer, follow the links, but just to reiterate some of the contemporary arguments made for why non-slaveholding white Southerners should fear the end of slavery, and thus, support the Confederacy: it would usher in a race war, it would bring down wages of white people who now would have to compete with formerly enslaved black people for jobs and who would work cheaper, the black South would align politically with the North against the white South so that the white South would no longer have any political power in the state or federal government, it would usher in a mixed race society and "degrade" the white South to the same level as formerly enslaved black people, and so on.

If this seems unconvincing, I would also add that it's pretty routine, even today, for people to be invested in political issues that don't always affect them directly. Think about issues such as immigration or abortion. It's pretty clear that people who aren't immigrants and don't know any immigrants personally, or people who can't have abortions, have very strongly and sincerely held beliefs on these issues. Given a generation of hot-headed propaganda on such an issue (as happened in the lead-up to the U.S. Civil War), a considerable proportion of these people would gladly volunteer their service to aid the cause, if a war began over a strongly-held belief such as these.

Further, consider more settled issues like women's suffrage or an end to Jim Crow. Why would a man ever support a woman's right to vote, when it would diminish his own political power? Why would a white person ever want to end Jim Crow, when it meant they no longer got the preferential treatment at "separate but equal" facilities?

Or even just think about the start of World War II. Why would British and French people care about Germany invading Poland when they aren't Polish?

It is quite routine for people to take stances on issues that, on the surface, might not seem to be in their personal interest. In fact, it's a quite well-studied phenomenon, of the psychological effects of partisanship and how partisan leaders can convince (often quite easily) their followers to support violence and wars to further political goals that don't always directly benefit the followers.

Some recent articles that generally explore the phenomenon include "Explaining Public Support for the Use of Military Force: The Impact of Reference Point Framing and Prospective Decision Making" by Hector Perla Jr., or "War, Manipulation of Consent, and Deliberative Democracy" by William S. Lewis.

One other recommendation is the book With Ballots and Bullets: Partisanship and Violence in the American Civil War by Nathan Kalmoe, and published by the Cambridge University Press. This should address some of your questions concerning the U.S. Civil War specifically.

I also recommend consulting with the sources linked in my previous answers above, which will address more directly why non-slaveholding Southerners supported slavery and the Confederacy.

There are also some other great and pertinent answers in this sub written on the topic of white Southerners' support for slavery and the Confederacy, but I'm on mobile now, so I can't link them all. But I will link this answer by /u/jschooltiger, who answers it from the perspective that Lost Causers often deflate the numbers of slaveholders in the South and/or try to diminish the personal and direct connection that most white Southerners had to slavery. Slavery was very, very widespread in the Confederate states.

So, not only is it a pretty easily disproven myth that war supporters must have a direct connection to the cause they are fighting for (very often, this is not at all the case), but specific to the Civil War, the disconnect between slavery and the average white Southerner is often overstated. The direct connection to slavery in the South was incredibly deep, and inescapable almost everywhere. And the places where it was escapable tended to be the enclaves of Union war support, such as in West Virginia, East Tennessee, and some other parts of Appalachia.