The ratifying states knew to a large extent what they were getting into. They were not merely confederates anymore. Rather, they were surrendering enormous amounts of sovereignty to a national government. This allocation of powers was formalized in enduring law and institutions, leaving little room to argue that the union existed only as a discardable convenience.

I say "to a large extent," though, because the constitution was still quite vague in many ways. The document identified broad domains of power but often did not get into the particulars. This ambiguity was so expansive that even men who worked on the Constitution walked away from it with very different ideas about US federalism. Hamilton saw federal supremacy in the Constitution. Jefferson saw a federal government that was (and should be) narrowly construed. The vagueness allowed different interpretations of what federalism entailed. States like South Carolina to saw the union as existing only between states and therefore something that they, as a state, had a complete right to exit. When that state did eventually secede, it said "that the Union now subsisting between South Carolina and other states under the name of the 'United States of America' is hereby dissolved."

I'm going to paste over here the response I gave in /r/USCivilWar:

Your question brings up an extremely important facet of the Civil War that often gets overshadowed by the larger slavery question (which is itself absolutely central to understanding the causes of the war), and that is how the Civil War was a constitutional crisis. While the South was very heavily invested in maintaining its slave institutions, its interpretation of the legacy of the American Revolution and the meaning of the Constitution and its founding was a cornerstone of its overall political vision. See, issues like states' rights touch on fundamental questions of American citizenship and legal legitimacy, specifically the notion of sovereignty. For the purposes of this discussion, sovereignty defines where the ultimate legal authority rests within the United States. Was it the people themselves or was it the states, acting on behalf of the people? This is a little question with huge implications and, as you might expect, Americans often disagreed on that point, from 1783 right up through 1861.

Next, I need to set the context in which the Constitution itself was ratified. I won't get into all the details or why it was deemed necessary, because what's important here is how incredibly precarious the ratification process was. Today nearly everything thinks the Constitution is a great idea and the implicit assumption is always that everyone loved the idea and jumped in with both feet. The truth is far less linear. There were some states that ratified the Constitution unanimously, such as Delaware, New Jersey, and Georgia. Others weren't unanimous but there were clear majorities, like S. Carolina, Connecticut, Maryland, etc. But here's where it gets tricky, because even if the requisite 9 of 13 states ratify, everyone knows that unless certain big states voluntarily sign up, the Constitution will likely fail in practice. Those states were Virginia, Pennsylvania, New York, and Massachusetts. Virginia ratifies by a margin of just 10 votes. Pennsylvania: 23 (phew!). Massachusetts: again just 10 votes. New York: 3 measly votes. Translation: ratification was a close call (source for those numbers is wikipedia) Translation: ratification was a close call.

Ok, why am I boring you with vote tallies? Because it's important to remember that getting the Constitution ratified was hard work and never a sure thing. It has to be sold to the various states to persuade them to sign up, and salesmen tell people what they want to hear, or at least phrase things in a way that let people infer what they like. This is very well demonstrated in the Federalist Papers, a series of documents written by James Madison, Alexander Hamilton, and John Jay to try and convince Americans of the merits of ratifying the Constitution. Here are a few relevant excerpts (though I cut out some superfluous text).

From Federalist No. 44, regarding the proposed Supremacy Clause of the Constitution:

"This Constitution and the laws of the United States... shall be the supreme law of the land, and...any thing in the constitution or laws of any State to the contrary notwithstanding." The indiscreet zeal of the adversaries to the Constitution has betrayed them into an attack on this part of it also... To be fully sensible of this, we need only suppose for a moment that the supremacy of the State constitutions had been left complete by a saving clause in their favor. In the first place, as these constitutions invest the State legislatures with absolute sovereignty, in all cases not excepted by the existing articles of Confederation, all the authorities contained in the proposed Constitution, so far as they exceed those enumerated in the Confederation, would have been annulled, and the new Congress would have been reduced to the same impotent condition with their predecessors. In the next place, as the constitutions of some of the States do not even expressly and fully recognize the existing powers of the Confederacy, an express saving of the supremacy of the former would, in such States, have brought into question every power contained in the proposed Constitution. In the third place, as the constitutions of the States differ much from each other, it might happen that a treaty or national law, of great and equal importance to the States, would interfere with some and not with other constitutions, and would consequently be valid in some of the States, at the same time that it would have no effect in others. In fine, the world would have seen, for the first time, a system of government founded on an inversion of the fundamental principles of all government; it would have seen the authority of the whole society every where subordinate to the authority of the parts

In other words, here it's being said that absolute state sovereignty would undermined the legitimacy of the national government to the point of non-function, and thus can't be present. The sovereignty of the people is more directly affirmed in Federalist 46, but pay attention to how the author in this case keeps trying to save face with advocates of state authority:

The adversaries of the Constitution seem to have lost sight of the people altogether in their reasonings on this subject;...They must be told that the ultimate authority...resides in the people alone... Truth, no less than decency, requires that the event in every case should be supposed to depend on the sentiments and sanction of their common constituents. Many considerations...seem to place it beyond doubt that the first and most natural attachment of the people will be to the governments of their respective States.

Later on in the same document:

It was...invariably found...that the attention and attachment of the people were turned anew to their own particular governments [ the states]; that the federal council was at no time the idol of popular favor... If, therefore, as has been elsewhere remarked, the people should in future become more partial to the federal than to the State governments, the change can only result from such manifest and irresistible proofs of a better administration... And in that case, the people ought not surely to be precluded from giving most of their confidence where they may discover it to be most due; but even in that case the State governments could have little to apprehend, because it is only within a certain sphere that the federal power can, in the nature of things, be advantageously administered.

Later on, same document:

It has been already proved that the members of the federal will be more dependent on the members of the State governments, than the latter will be on the former. It has appeared also, that the prepossessions of the people, on whom both will depend, will be more on the side of the State governments, than of the federal government. So far as the disposition of each towards the other may be influenced by these causes, the State governments must clearly have the advantage.

Also in Federalist 46 (though the quotes are far too long to try and break down here), it is assured that the federal government will always be subordinate and forced to reflect state interests, and that states will remain the primary vehicles for expressing the will of the people. Should any gross infractions of the federal government arise, it is assured that the states will retain the will of the people and be able to overturn any federal infractions through sheer force of popular will.

Between these types of defenses of the proposed Constitutions and others, it became clearly possible to envision the Constitution as a voluntary association of sovereign states or to envision it as the establishment of a national government based on the sovereign will of the people. The Federalists were perfectly happy to let different Americans reach different conclusions if it led to ratification. This left fundamental questions about the nature of the Constitution unanswered, and ripples of those questions can be seen throughout the antebellum era. Notice, though, that the ability of a state to separate itself from the Constitution once ratified is not expressly stated, but neither is it expressly denied. Numerous claims would be made by different Americans throughout the antebellum era within this ambiguous space.

Take, for example, the later Virginia and Kentucky Resolutions, penned by Jefferson and Madison. These documents were very controversial and specifically claimed that states have the power under the Constitution to judge the constitutionality of federal laws and even to declare them void. They also specifically described the Union as a "constitutional compact," implying that it is a contract (and contracts can be declared void in certain circumstances). Once again, though, these resolutions do not specifically claim the power of states to leave the Union under certain (or any) circumstances.

Other events, like Supreme Court cases like McCulloch v. Maryland specifically affirmed the power of the federal government over state action or resistance, though they once again left the ultimate question unanswered.

The fact that these questions are left ambiguous, and that very well crafted arguments can be made by each side, is the whole point of this rather long post. Some people did affirm the ability of the states to leave the Union, while others opposed it, based on how they interpreted the original purpose of the Constitution. This brings us right up to Nullification.