(1/2)
It's slavery, but explaining how and why takes a little more than just saying that. :)
From the emergency of the two-party system in the 1830s -or re-emergence, if you want to count the Federalists and Republicans- both the Democracy and Whiggery were genuinely national. They fielded candidates and had enough success in both the slave and free states to at least provide serious competition to each other. The Democrats are usually on top, but the Whigs have significant breakthroughs that could have been consolidated had they enjoyed better luck with presidents. It's true that their constituency was somewhat more northward-facing, but Louisiana and Georgia were pretty good slave states for Whigs and they did well in the Upper South.
Doing well in Southern politics means a minimum of proving that your party is safe for slavery, the central, overriding concern of the section since at least 1800. What that in turn means is that your party has got to be able to frustrate and sideline antislavery members inside the coalition. Otherwise your opponents will cast you as fully in bed with them on destroying slavery, provoking a race war, financial ruin, and so forth. The fears of all this can read as irrational, but they're not. White southerners literally live intimately among people that they subjugate and torture for a profit.
Through the 1830s and into the 1840s, the Whigs do a fairly good job of proving they are safe for slavery. Texas annexation, the Mexican War, and the "Compromise" of 1850 change all that. Going into great detail here would take pages and pages and be basically the domestic political history of the United States from 1836-1860. I can do that on follow-ups -I am very boring- but for this post I'll focus on the big picture stuff.
Texas
When Texas wages its successful defense of slavery and becomes a largely unrecognized independent state, with visions of its own slaveholding empire genocided away from the Native Americans who had the temerity to live nearby dancing in their white heads, they first want annexation to the US. Most white Texans were until quite recently Americans and white Americans, particularly white Southerners, have a keen interest in the Lone Star Republic. Joining up is the obvious path forward. Andrew Jackson and then Martin Van Buren, both proslavery men -Van Buren later changes his mind on that to revenge himself on the Democracy for denying him a second presidential nomination- want no part of Texas. They understand that a huge enslaved territory being added to the Union is political poison. It's a massive gain for the South with no real compensatory gain or even fig leaf for freedom. Their coalition, like the Whigs', requires support in both sections. That in turn requires either silence about slavery or at least plausible deniability when advancing it. Most white Americans, being massively racist, are happy with the latter. A minority, while still almost all being massively racist, are not. Bring slavery into the spotlight and the vague discomfort that many white northerners have with slavery could move them to the second column.
The Whigs are mostly against annexation for similar reasons, but their antislavery wing is a big stronger as a result of their coalition skewing more Northern and they have a generally lukewarm program on territorial expansion for a combination of cynical and ideological reasons. On the first, new territories tend to become Democratic states. On the latter, the Whigs are massively into reform and improvement of white civilization. They want a more morally uniform and economically forward state, buttressed by tariffs and infrastructure projects for the betterment of domestic commerce. The Democracy is much less interested in moral reform and more wedded to a specifically agricultural and agrarian vision of society wherein frontier life is more virtuous in itself.
Annexation thus stalls out in the 1830s and the Texans lost interest for a little while as they embarked on the planned genocide. It comes back in the 1840s when the mood in Texas shifts and the John Tyler wants to win a second term. Tyler is a Southern Whig who is basically not a Whig. He was Harrison's VP, then Harrison went and died. That left Tyler, a proslavery extremist, in the big chair. He proceeded to feud with his party and pledge himself opposed to almost everything any of them stood for. They read him out of the party. Tyler in turn buddies up with other proslavery extremists like Abel Upshur and John C. Calhoun. He needs a big win if he wants to get his own term in 1846 and Texas is both ideologically congenial to him and genuinely popular with a lot of ordinary people.
Tyler doesn't get it, then he gets it, then he loses anyway. He's able to secure a treaty of annexation, but it can't get through the Senate because it needs 2/3. He doesn't get any nominations to the presidency because the Whigs don't want him, the Democrats have their own guy who is also a proslavery guy and one of them, and his attempts to form his own party go nowhere. So Tyler doesn't get Texas and doesn't get the presidential term he wants. But in the end he gets a joint resolution of Congress through with a simple majority that annexes Texas. This was widely considered unconstitutional and shady as midnight on Pluto in the winter, but: free Texas. The Whigs ran Henry Clay (again) in 1846 and his shifting positions on Texas contribute to his losing. Polk wins, but Tyler gets the joint resolution through before his term expires.
This is a mixed bag for the Whigs. Texas hurt them on a southern priority, but it's also a party priority and the southern Whigs who already exist don't seem to have suffered too badly from it. What's much more damaging is what it sets off.
The Mexican War
A reason that annexation stalled previously which I haven't touched on is that Mexico doesn't really agree that Texas is independent and even if it is, does not agree about where Texas ends and Mexico begins. The Texans claim the Rio Grande as their border, which gives them both the border we would recognize on a map to the South and also half of New Mexico and substantial chunks of other western states. The government in Austin controls basically east Texas, with a line of actual control somewhere around the Nueces River and then fading off fast into land claimed by Mexico but often controlled by Native Americans to the west. An expedition to exert control over Santa Fe got lost, ran short of food, and basically got arrested.
Annexation was rightly understood as something that would lead directly to war, which James K. Polk engineered to commit Grand Theft: American Southwest. From the beginning Americans figured they would win it, which they did. That meant they started planning for the peace well in advance, both in terms of the real estate shopping list and what to do with it. An antislavery congressman, David Wilmot, proposes that slavery be banned on any land taken from Mexico except for Texas to keep it lily white, basically. Cue massive outrage that goes on for years. To Southerners, this is being hedged out of land they're literally bleeding for at that moment. (Slavery is coterminous with white Southern identity in the antebellum. If they can't take slaves, they aren't allowed to go.) There's a bunch of constitutional theory ginned up against it, but the main thing is they think slavery is good and should expand. Wilmot and those who vote for the proviso, which is pretty popular among both parties in the North, think it's not and should not.
Complicating all this, the gold rush gives California enough white inhabitants that it needs a real government ASAP and the white people there decided they didn't want slavery. Thus a dispute that's intense but basically abstract becomes real and the white South is faced with losing its parity in the US Senate to go with its minority in the House. The future security of slavery would thus hang entirely on winning presidential elections in an environment where the Northern population and thus electoral college margin is growing faster than the South can.