So, political power was ABSOLUTELY a major factor in this issue, and you bring up the point that virtually every researcher has to tangle with. We as academics have come up with a few answers that are a little bit weird, and the explanations can be very in-the-weeds based off of how these folks perceived the Constitution and the Bill of Rights the role of government as a whole. This is a really complex issue, and the amount of scholarship on it is like, overwhelming. It is also a super fair question because the South was basically a toddler at this point and freaked out every five seconds, so the North and West were constantly just giving them what they wanted to prevent an all-out tantrum.

Alright, so, to put it simply, the South's conception of the role of government was that the government existed for the richest of people. Why does that matter in this context? Well, the richest Americans at this time - especially in the South - were enslavers who owned massive plantations and could claim ownership of a hundred or more people. When you consider how much the average enslaved person was sold for in 1860 (1000 dollars), some of the wealthiest enslavers were worth more than 3 MILLION dollars in todays money in just folks held in bondage. That is not including their plantation homes, the land, artwork, or anything else. It is truly preposterous the kind of money these people were worth.

At a certain point, many of these folks began to have too much money to even know what to do with. Land was either unusable for plantation agriculture (this was the case in peninsular Florida and some of the Mexican Session land - think of Arizona), already occupied by plantations, cities, or yeoman farmers, or it was out West or somewhere outside of the United States (see: the Caribbean and Latin America). These people had become massively wealthy through land speculation and enslavement, so a lot of them understood the expansion of slavery to be a means for continued revenue sources. This was basically the Southerner's conception of the American dream - to secure land, revenue sources, and assets which could be liquidated for your children. It was that deeply ingrained in Southern society.

So, back to government: Southerners understood government as a vehicle to protect these notions of the American dream. They understood the role of the Fourth and Tenth Amendments as essentially efforts to protect slavery - the Fourth protects the right to property (and of course enslaved property is, in the minds of Southerners, the most important type of property) and the Tenth basically limited the scope of the federal government. So, in the South's eyes, the federal government was violating the Constitution by making any efforts to abridge the expansion of slavery. These were PART of the grounds for the majority opinion of Scott v. Sandford (1857), which stated that Congress had no right to make any legal limitations to slavery in the territories. When they saw the violence of Bleeding Kansas and John Brown's raid at Harpers Ferry, VA, along with the pervasive belief that abolitionist conspiracies were actively trying to destroy the South anyway, they truly believed that their way of life was under threat.

Sources:

Property Rights in Slavery and the Coming of the Civil War by James Huston

Ideological Fruits of Manifest Destiny: The Geopolitics of Slavery Expansion in the Crisis of 1850 by Major Wilson

"They Are Very Much Interested in Obtaining an Unlimited Slavery": Rethinking the Expansion of Slavery in the Louisiana Purchase Territories, 1803-1805 by John Craig Hammond

Slavery and the Oregon Territorial Issue: Prelude to the Compromise of 1850 by R. Alton Lee