They stayed, but there was quite a lot of animosity.

The TL;DR version is that the English didn't always honor the surrender agreement, the Netherlands briefly took control back of the colony in 1673 before giving it up again, and the local Dutch citizenry of New York rose up and rebelled in 1689 in an event known as Leisler's Rebellion. The rebel leader, Jacob Leisler, and his son-in-law were controversially put to death, and were the only casualties of the Rebellion. New York politics was largely divided along pro-Leisler and anti-Leisler lines for the next 20+ years. By the 1720s, however, the Dutch population's political control began to fade, as the mix of New York's diverse population began to change. The Dutch who remained in the Five Boroughs began to move to the country, and they remained an intact minority community until the Revolution. But the American Revolution disrupted these communities with English speaking war refugees, and with the U.S. government opening up the west for expansion, these Dutch speaking communities had given way to English by the Civil War. A small community remained up until the early 20th Century. The last "Jersey Dutch" native speaker died off some time after 1911--probably in the 1920s or so.

The longer version goes like this:

The Dutch had been nervous that there would be an English attack for many decades. The border of New England and New Netherland had long been in dispute. They both claimed all of Long Island, and both had settlements on it.

Despite some preparations, the Dutch were caught off guard when the English arrived in the Bay of New York on August 27, 1664, with four ships and many hundred fighting men. The force was large enough that there wasn't much chance of New Amsterdam to defend itself. It was either surrender, or suffer massive casualties that will ruin the colony, and then surrender.

Still, the Director-General who ran the colony, Peter Stuyvesant, didn't want to give up, and it took two weeks of negotiations before he would. On September 8, 1664, he finally agreed to surrender terms. For the Dutch living in New Netherland, he seemed to get them a pretty good deal. He negotiated that the citizens be allowed to continue the policy of religious tolerance, and allow the current setup of Dutch Reformed, German Lutheran, and English Quaker churches/houses of worship to remain. They were also allowed to retain their system of local government, with its burgomasters, schepens, and schouts being renamed mayor, sheriff, and aldermen, respectively. And the property rights of any landholder would be recognized, so long as the person swore allegiance to the King of England.

Almost immediately, however, in the eyes of the local Dutch citizenry, the English started reneging on the deal. Within the first year, they stopped allocating tax money to the Dutch church, which ran the Dutch schools that most of the colony's children attended. Not only that, the new government started demanding that the Dutch pay taxes toward the English church instead. The Dutch adamantly resisted to no avail.

Further, while the English allowed the Dutch structure of government to remain, they immediately instituted that all official government communications, including court sessions, be conducted in English. This was a language which very little of the population could speak or understand, let alone write. In those days, court sessions had an added importance, because there was no real patrolling police force. If somebody wronged you, your recourse was to show up to the weekly court session and make a statement, and the court would then compel the other person to respond. Now, the citizens of newly renamed New York couldn't do that in their own language.

The resistance to these changes was strongest among the Dutch farming class, but even among the Dutch elite, there were misgvings. In 1668, Jeremias van Rensselaer, one of the richest people in the colony, wrote to relatives back in the Netherlands:

"Now it seems that it has pleased the Lord [to ordain] that we must learn English. The worst of it all is that we have already for nearly four years been under this jurisdiction and that as yet I have learned so little. The reason is that one has no liking for it."

All was not lost for Dutch New Yorkers. In 1673, in the midst of the Third Anglo-Dutch War, Dutch naval Captain Cornelis Evertsen the Youngest retook control of the colony with a force of 23 warships. This was great news to the local Dutch citizenry, who still made up a majority of New York's diverse population, despite the more recent English settlement in the area mostly from New England transplants.

The Dutch government itself was a little bit pissed off because Evertsen had actually been tasked with capturing French Guinea in South America, or else the island of St. Helena off the coast of Africa. With the peace treaty that ended the war in 1674, the Dutch government relinquished control of New York for the final time, and in exchange the English gave them Suriname.

It should be said that there was a lot of bad blood between the English and Dutch at that time, as this quite funny piece of propaganda from the period illustrates. This Dutch/English anatagonism carried over to New York, both for political reasons but also out of the personal experience of Dutch New Yorkers dealing with the local English government.

With the restoration of English rule in 1674, the antagonism only got worse. Brooklyn was a particular hotbed of it, which is why the English government demanded that the Dutch in Kings County all once again take a public Oath of Allegiance to the King of England in 1687.

Another issue that came up is that, in the 1680s, the English government began forcing the Dutch population to drop their traditional "Dutch names" and start using English-style fixed surnames. The Dutch very much resisted this, and in the period of 1680-1710, there are ample instances of Dutch people being forced to use one name on English government documents while they continued to use their original Dutch name on baptism and marriage records in the Dutch church.

When William and Mary ascended to the English crown in early 1689, the Dutch New Yorkers celebrated since William was the Dutch Prince of Orange. But this didn't much affect the way the English government in New York handled the Dutch, and things quickly went back to the way they were.

All of this culminated in Leisler's Rebellion in 1689, in which German immigrant Jacob Leisler led a force of Dutch farmers to take over the fort at the lower tip of Manhattan by force (though nobody was killed). The fort also contained the treasury of the colony, so the Dutch controlled most of New York's finances.

The English government had to call on the Navy to put down the rebellion, which they did peacefully. Yet, despite the fact that nobody had been killed during Leisler's Rebellion, Jacob Leisler and his son-in-law were controversially put to death in 1690.

The result was that, for the next two decades, New York politics were very much organized around two political parties: one of "Leislerians" and another of "anti-Leislerians". Essentially, poor Dutch farmers versus everyone else.

But by the 1710s, enough English immigrants had made their way to New York that the Dutch began their slow descent to minority status. Manhattan and then western Long Island began to be overwhelmed with English speakers, and the Dutch began to move away from (what are now) the Five Boroughs. Some of them went to the eastern end of then-Queens (Oyster Bay, Hempstead), while many more settled along the Hudson River Valley between Westechester County and Albany, or the Passaic Valley in northern New Jersey.

The Dutch community survived in those places outside the city in fairly considerable numbers until the end of the 1700s, despite the English/British continued efforts to devalue them. The Dutch population were often looked down upon as stupid, lazy rednecks who smoked too much, drank too much, were dishonest, and couldn't speak proper English. This is exemplified by Washington Irving's "Rip Van Winkle", or this old joke published in the Poughkeepsie Journal in 1849, or the idiom "in Dutch" meaning "in big trouble".

Nonetheless, Dutch New Yorkers remained a proud and intact community until the American Revolution. But early in the Revolution, NYC was occupied by the British, which resulted in a large population of English speakers from Manhattan and Long Island becoming refugees in the Dutch towns of New Jersey and especially the Hudson River Valley, disrupting the existing communities. At the end of the war, not all of these English speakers left these now-mixed towns, and the new U.S. government opened up the Western frontier for settlement, so many young Dutch people left for better opportunities.

Shortly after 1800, the Dutch churches in the United States no longer required the sermons to be conducted in Dutch, a transition that several of the churches had recently made. Within the next half century, the Dutch communities of New York and New Jersey had all but disappeared, although it wasn't until after 1900 that the last of the "Jersey Dutch" speakers in northern New Jersey died off. An article in the December 5, 1911, issue of the Asbury Park Press mentioned 200 Jersey Dutch still left, all over the age of seventy. There hasn't been a mention of them since.