I don't have a clear source on the Constitutional Convention itself, but the subject of inserting official language about secession into the Constitution was a definite feature of some of the state ratifying conventions.
The most notable of which is the Poughkeepsie Convention in New York. Anti-federalists had been pushing to insert such language into the Constitution. Amar elucidates:
But exactly how were these states united? Did a state that said yes in the 1780's retain the right to unilaterally say no later on, and thereby secede? If not, why not?
Once again, it was in New York that the answer emerged most emphatically. At the outset of the Poughkeepsie convention, anti-Federalists held a strong majority. The tide turned when word arrived that New Hampshire and Virginia had said yes to the Constitution, at which point anti-Federalists proposed a compromise: they would vote to ratify, but if the new federal government failed to embrace various reforms that they favored, "there should be reserved to the state of New York a right to withdraw herself from the union after a certain number of years."
At the risk of alienating swing voters and losing on the ultimate ratification vote, Federalists emphatically opposed the compromise. In doing so, they made clear to everyone -- in New York and in the 12 other states where people were following the New York contest with interest -- that the Constitution did not permit unilateral state secession. Alexander Hamilton read aloud a letter at the Poughkeepsie convention that he had received from James Madison stating that "the Constitution requires an adoption in toto, and for ever." Hamilton and John Jay then added their own words, which the New York press promptly reprinted: "a reservation of a right to withdraw" was "inconsistent with the Constitution, and was no ratification."
Amar, NY Times(wayback machine version if NY Times paywall is a problem)
Amar, America's Constitution: A Biography
Attempting to address the specific context you might be talking about, Madison did briefly address something arguably secession-related during the Convention:
...The last clause of Resolution 6. [FN11] authorizing an exertion of the force of the whole agst. a delinquent State came next into consideration.
Mr. MADISON, observed that the more he reflected on the use of force, the more he doubted the practicability, the justice and the efficacy of it when applied to people collectively and not individually. -A union of the States containing such an ingredient seemed to provide for its own destruction. The use of force agst. a State, would look more like a declaration of war, than an infliction of punishment, and would probably be considered by the party attacked as a dissolution of all previous compacts by which it might be bound. He hoped that such a system would be framed as might render this recourse [FN12] unnecessary, and moved that the clause be postponed. This motion was agreed to nem. con.
http://avalon.law.yale.edu/18th_century/debates_531.asp
Here Madison acknowledges that using force against a 'delinquent state' as something to be abhorred, and hoped to prevent the need to resort to any sort of activity by framing systems to avoid it. This is not the same thing as discussing (much less approving) the right for secession. In fact, this seems to argue the opposite: that using force on a state will make it secede, not the other way around.
Madison's very concerns here were later articulated in 1861 when Lincoln had to delicately handle Fort Sumter without unduly being seen as the provocateur.