even after the constitutional convention reformed the government in 1789, many peoples' loyalties were to their home states rather than the national government. (you can see this during the civil war, when countless southerners decided that their principal loyalty was to their state rather than to the nation.) national identities don't develop overnight-- and a national identity hadn't really developed during the early republic in the way we think of it now.
before 1776, there was no single institution in north america with authority to govern the colonies as a whole. all 13 were founded independently, and developed in parallel, but there were no common institutions of government except the toothless continental congress.
tellingly, the colonists chose the word "congress" to describe their new legislature, rather than "parliament," "diet," or "assembly." "parliament," "diet," or "assembly" would imply that it was one country, under one sovereign, as in britain or the holy roman empire. the word "congress," in the traditional sense, implies that the attendees are repping their individual, sovereign states, as in the congress of vienna in 1815. much like the modern european union, the 13 states maintained very strong local identities, and were loath to give up any authority to a faraway national government. the early republic, in other words, was a confederation of independent states, united voluntarily for the purposes of common defense and foreign relations. the confederation had no independent taxing power, and was totally reliant on the states to implement its laws. the union was perpetual as a matter of law, but the national government had no power to force a state to stay.
at the constitutional convention in 1787, a few radicals, like alexander hamilton, actually proposed transforming the states into instruments of the national government, much the way that the later departements of france are simple instruments of paris. (the independent authority enjoyed by the proconsuls of rome or the satrapies of persia only led to rebellion and disunity, in his opinion.) needless to say, this argument didn't fly, and the states' independent sovereignty was kept.
it was a much narrower thing than you'd expect when it came time to ratify the constitution. the anti-federalist party had many, many valid criticisms of the proposed constitution. it took the promise of reforms, and a bill of rights, to even secure the approval of the constitution.
post-constitution, the u.s. became a federation, where sovereignty resides both in the national government and the state governments. again, though, identity doesn't move nearly as fast as law does. 87 years later, local identity was still strong enough in the south that countless americans chose to commit treason en masse rather than betray their states.