There was an immense amount of dissent regarding the idea of secession, especially in the Appalachian regions of the Confederacy. This was in large part motivated by the general neglect of the area by the landed gentry that traditionally dominated Southern politics. In addition, there was sentiments that betraying the Union meant a loss of economic security as well as a feeling that to fight against the country your fathers and grandfathers fought for was wrong.

To give you an idea of how far this went, close to 100,000 white Southerners fought in Union regiments, as well as the additional large numbers of ex-slave regiments. This doesn't count the large numbers of pro-Union partisans that fought a running guerilla war through much of the upper south, especially in Northern Alabama and Tennessee.

This actually played a pretty significant role in the defeat of the Confederacy, as, unlike the Union, it was basically fighting two wars. First and foremost, that with the Union, but also a secondary civil war with itself that consumed a large amount of fighting men that were desperately needed elsewhere.