Was it really worth starting a war just to avoid paying their workers one cent? Wouldn't a war inevitably be more expensive and destructive than abolition?
TL;DR: it wasn't about the money. It was about social structure, or from a different direction, racism.
Lincoln floated a compensated-emancipation plan in 1862. He addressed border-stage congressmen first, then Congress at the end of the year, pointing out
The border-state congressmen turned him down. (A minority just said that they'd ask their constituents.) They argued that their people couldn't pay such taxes as would be needed -- ignoring the fact that the Union as a whole would be paying, so their states would be net gainers.
The key, I think, is in the part after the cost calculations (which they slanted by including Tennessee and Virginia, rebel states, which the President did not include). They pointed out that they had a right (which the proposal was acknowledging by offering compensation) and "We do not feel called on to defend the institution" (which defence Lincoln wasn't asking for).
More seriously, they twice referred to complying as "a sacrifice". They analyze Confederate society as having disunionist / state sovereignty elements, but also those who strove "to maintain and preserve its rights of property and domestic Safety". So the only way to split them would be to reassure the second class that their human property was safe (when Lincoln got a request to say it before his inauguration, he wearily replied that he had said so on so many occasions that he would have looked like an idiot to repeat it yet again). They refer to "the triumph of a principle which the people of the Southern states regard as ruinous to them". They later call it "extra territorial interference" (it was an offer of voluntary compensated emancipation, to the state(s) who decided to emancipate).
If I can be jocular, it wasn't about accepting benjamins in their pockets but keeping Uncle Tom in his cabin. It was social structure -- slavery simply as slavery was a thing of great value to them.
McPherson's Battle Cry of Freedom covers it, of course. If you want to see original source documents -- Lincoln's message to border congressmen, their two replies, his message to Congress -- you can see them here.
To add to OP: I assume slavery had widespread support among poor whites, why was this? It would seem to me that slavery stood as unbeatable competition to white wage-workers.
Because, slaves were fairly profitable in the labor that they produced as well as being worth a very significant amount of money themselves. In 1860, the South held somewhere around 4 million slaves. If slaves averaged around $800 each (they ranged from a few hundred to a thousand dollars in 1860 dollars), that comes out to $3,200,000,000 in 1860 dollars. In 2015 dollars, that is $86,400,000,000.
That is just the cost of the slaves themselves. That doesn't count that these folks served as the entire foundation of the economy of the South.
They largely recognized it as a "necessary evil" in terms of the basis of the Southern economy (however, they for the most part had no problems morally in keeping slaves). However, it was just too entwined in both their economy and their hold on the social caste system to do anything about it without causing tremendous upheaval.
TD:DR They didn't want to end slavery, but even if they did, there was no practical solution for them to do so without completely destroying Southern society.