Excellent question that I think should be broken down into its two parts:

1) Why were Southern slave owners unbending on the issue of slavery? This is a difficult question; however, I would argue you need not look any further than Vice President of the Confederate States of America Alexander Stephens' famous "Cornerstone Speech" delivered in Savannah, Georgia 1861. In describing how the government of the CSA differed from that of the Union, Stephens states, "Our new government is founded upon exactly the opposite idea; its foundations are laid, its corner- stone rests upon the great truth, that the negro is not equal to the white man; that slavery -- subordination to the superior race -- is his natural and normal condition." It is important to note that both slaveowners and non-owners embraced this rhetoric equally. Southerners' insistence on slavery was due in large part to their ideological understanding that Africans were unfit for freedom. The idea of a substantial free African population, even if kept as second-class citizens and barred from the political arena, posed a serious threat to all white citizens. For the adamant defender of slavery, the real injustice for Africans was the state of freedom.

2) Why wouldn't Southern slaveowners have considered a British style emancipation scheme? First, when comparing American slavery to British slavery in the Caribbean, there are some striking differences. Foremost, by the nineteenth century American slavery was spreading rapidly to the deep south and westward with the expansion of "King Cotton." Regions of Georgia, Alabama, Louisiana, and Texas were ripe for growing cotton, which was in high demand throughout the industrial world. Although the transatlantic slave trade had ended, the domestic trade was bring slaves "down river" from the Old South to the Cotton Kingdom. Large plantations were springing up and the slave population was on the rise in the United States. In contrast, British slavery in the Caribbean was on the outs as it was largely stagnant and continued to produce commodities including sugar that were declining in value. If you are to follow the William's thesis, beginning in the late eighteenth century, the British had reinvested their mercantile profits in manufacturing meaning that they were moving away from the production of sugar in the Caribbean while still relying heavily on slave-picked cotton from the United States. Secondly, the vast majority of British plantation owners practiced what was called "absenteeism," meaning that though they owned large plantations in the Caribbean, they resided in the United Kingdom and their plantations were run by overseers. Though they stood to lose a substantial amount of property, they would not be forced to live among free populations of Africans in the same way American southerners would have. Finally, following American Independence, the abolition movement in England gained tremendous momentum. William Wilberforce and Thomas Clarkson built strong abolition societies that illustrated the evils of slavery and argued that the British Empire was above the barbarous behavior practiced in the Americas. TL;DR: American Southerns (both slaveowners and non-owners) had both an ideological and economic commitment to slavery that made the "particular institution" a nonnegotiable during the mid-nineteenth century. British emancipation worked in the Caribbean because the plantation economy of the British operated differently than that of the American South.

Sources: Eric Williams, Capitalism and Slavery

Sidney Mintz, Sweetness and Power

Walter Johnson, Soul by Soul and River of Dark Dreams

Eugene Genovese, The World the Slaveholders Made

Edmund Morgan, American Slavery, American Freedom

Christopher Leslie Brown, Moral Capital

The practical difficulties of compensated emancipation for the antebellum South are insuperable even if they wanted it, which I just wrote about a bit last week so I'm going to shamelessly piggyback that. :)

But you're right to note that they chose against it, whether the option was realistic or not. The South's small government, strict construction orthodoxies were built around preserving slavery against a nation which might have the power to enact such a thing. A government that could living constitution in a national bank could then use that bank to get the funds to buy all the slaves. Even a budget surplus suggested the possibility of such action on a limited scale, which was enough to make it a nigh-existential peril in the Jackson years.

That's the what, extended. Here's some why:

Antebellum Southerners didn't believe they could do as well with free labor. The example of the British Caribbean in the years after emancipation was not at all encouraging from a profit standpoint, to the degree that elements of British opinion themselves believed it had been a mistake. In the white South, emancipation was largely viewed as a ruinous folly which the British were so keen to prosecute on others entirely to level the playing field and make up for their own grievous error.

The problem is economic, as it is with any scheme to actually do compensated emancipation, but it's got a still-uglier side. The model that white Southerners had of a free black population was Haiti, which they understood as both poor and prone to genocide against whites. Without slavery keeping them in check, the great numbers of blacks would obviously embark on some kind of war of extermination against the South's whites. They expected to win that war because they knew whites were just better at everything important, but also knew that it would be costly and brutal. They and their loved ones might not survive to enjoy the continent cleansed of blacks.

For indications that the local black population was just the same as the Haitian one, they appealed both to racial theory (black people are just like that, and these people are black here too) and to their own observations that free blacks were lazy vagabonds prone to crime, at best. Freedom might literally unhinge their minds, a claim they suggested that the 1840 census supported. That census showed that the free black population of the north was disproportionately resident in asylums or otherwise public burdens out of some disability. The truth is that the census was sloppily done and full of obvious errors, which Southerners would admit to on an individual basis, but they refused to contemplate corrections or a do-over because 'everyone' knew the central point was true anyway.

So the option of a changed social system isn't really on the table for them, even if they could be paid well. By 1860 (and probably by 1800) the white South is largely settled on the point that slavery is, at minimum, unfortunate but indispensible and increasingly (especially after the 1830s) that it's actually a blessing for whites and blacks alike. White people get freedom from class struggles that have become common in the North. Black people get educated in civilization, Christianity, and so forth. How much of either of those stances is just a public face or personal rationalization for what they know is brutal exploitation is something we could debate, but either argument ends up in preserving slavery for the foreseeable future.

With an antislavery president coming into office, soon to have the power of patronage to fill southern offices with either antislavery men or at least soft on slavery types who would surely try to build a southern antislavery party, the existential crisis has come. If Southern whites aren't united enough and firm enough, they will be swept away. Even if the Republicans did nothing, the mere ascendancy of their party conjured images of frightful slavery debates on the national stage that always seemed to somehow get back to the slaves and, so southern whites believed, give them ideas about uprisings which could in turn bring on that race war and financial ruin. There are some who argue that 1860-1 is possibly not the right time for secession, or that they should at least wait until Lincoln does something, but the longterm trend had been clear for about a decade prior: the South was increasingly the minority section and thus fundamentally insecure within the Union. Either radical revisions to the governing order of disunion had to come at some point. They risked everything either way, but saw the conquest of their section as extremely improbable, so disunion became the easier path.