Other scholars may know better than me, but I'll take a run at this.

First, we must keep in mind that the Federalist papers were public, political, and persuasive documents designed to help the Constitution get ratified. The Constitution was not a universally or even widely accepted proposition at first and passionate political debates took place over it. The Federalist papers were part of these debates, and therefore Jay and the other authors cannot be trusted to fully and fairly speak for the arguments of their opponents. They might exaggerate or misrepresent the other side to make them look less favorable. I expect that was happening here.

The Anti-Federalists were a looser group of opponents to the Constitution, whose writings simply weren't as cohesively organized as the Federalist Papers. Like many loyalists during the Am. Revolution, they weren't diametrically opposed to the concept of the United States, or the broader principles it was associated with, but rather disagreed on the best way forward.

Okay, enough preamble (heh). What I think Jay is doing here is not citing some mainstream Anti-federalist argument that the U.S. should be abandoned in favor of multiple confederacies, but rather arguing that any of their proposed alternatives to the Constitution, which would naturally be less centralized, would soon disintegrate into smaller confederacies. He's saying that only the Constitution can hold the country together.

Let's unpack that.

The U.S. won its independence, but that didn't make it a strong, autonomous country that could withstand foreign influence. There was a real and pervasive danger felt by Americans that they might be made puppets of larger foreign powers in Europe despite nominal independence. One major facet of this fear was the possibility that England might, say, pit New York against Virginia (or whatever) through side deals or other pressures, carving up the United States for its own purposes. David Hendrickson has interpreted the eventual Constitution as a kind of treaty between the states, designed to present a unified front for international protection as much as domestic government. Without it, they feared, the U.S. would devolve into a kind of competitive European-style state system, with shifting alliances and bids for power that ultimately hold everyone back.

The Anti-federalists shared this fear, and were no more eager to see the U.S. divided than anyone else. However, they opposed diminishing state sovereignty in order to provide that security, and feared the despotic potential of a strong national government. Shared democratic values and commercial ties, they argued, would be enough to hold the Union together. Conversely, they feared, strong coercive powers from a central government could drive political wedges between diverse states and actually cause the discord it was meant to prevent. To the Anti-federalists, in Hendrickson's words, "union could be acceptable only if it supported unequivocally the independence for which the states had fought. This kind of union [i.e. the Constitution], promised to subvert it."

They further denied that the threats to national unity were so dire that a strong central government was required in the first place. Patrick Henry decried the possibility of internal discord as a "hobgoblin" that had "sprung from the deranged brain of Publius" (the writers of the Federalist Papers). Anti-Federalist William Grayson of Virginia likewise criticized the idea:

We are now told by the honorable gentleman [Governor Randolph of Virginia] that we shall have wars and rumors of wars, that every calamity is to attend us, and that we shall be ruined and disunited forever, unless we adopt this Constitution. Pennsylvania and Maryland are to fall upon us from the north, like Goths and Vandals of old; the Algerines [North African, external threat, and a common proxy used by England/France], whose flat-sided vessels never came farther than Madiera, are to fill the Chesapeake with mighty fleets, and to attack us on our front; the Indians are to invade us with numerous armies on our rear, in order to convert our cleared lands into hunting grounds; and the Carolinians, from the south (mounted on alligators, I presume,) are to come and destroy our cornfields, and eat up our little children! These, sir, are the mighty dangers which await us if we reject--dangers which are merely imaginary, and ludicrous in the extreme! Are we to be destroyed by Maryland and Pennsylvania? What will democratic states make war for, and how long since they have imbibed a hostile spirit?"

Here we see examples not only of how Anti-federalists didn't typically want disunion and separate confederacies, but they still saw the proposed Constitution as an unnecessarily strong preventive measure, for which the cost (erosion of state sovereignty) was too high. We also see the nature of the discourse, and its tendency toward hyperbolic representations of the other side.

Therefore, I think Jay was not communicating an honest appraisal of his opponents' arguments but rather issuing an implied threat of sorts: that these Anti-federalists don't understand the stakes, and aren't willing to do what's necessary to preserve this Union. Follow them, and it will disintegrate.

Source for all this: David Hendrickson, Peace Pact: The Lost World of the American Founding