The Connecticut River had been the border claimed by the Dutch, but Connecticut sought to preserve the ports they had established on Long Island Sound, and in 1650 negotiated a border 10 miles east of the Hudson River—but the Dutch never ratified the treaty. Meanwhile, the 1667 Treaty of Breda gave New York to England.

Finally, in 1682 an agreement with New York (ratified in 1683) set the Connecticut/New York border 20 miles east of, and parallel to, the Hudson River. Connecticut also received the rectangular section of land east of the Byram River including Greenwich and Stamford, but had to give an equal amount of land back to New York further north along its western border, an area known as the “Oblong.”

The details are given in Boundaries of the United States and the Several States, a US Geological Survey book available as a PDF here. (p. 110) or in Mark Stein's book How the States Got Their Shapes. A Connecticut History webpage also covers the subject.