Do any other places on earth do this?

There's one example in Germany: Bremen, Bremen. Bremen (the city) is part of Bremen (the state), together with another city called Bremerhaven (roughly translates to Port of Bremen). This is the only example in Germany, for Hamburg and Berlin, there is no distinction between the state and the city.

The Dutch were originally in control of the city of New Amsterdam, which was part of the larger province of New Amsterdam. When the British took over in 1664, they renamed both the city and the province after James, the Duke of York (later King James II); for a short time, from July 1673 to November 1674, the Dutch recaptured and occupied New York, renaming it New Orange, but it was given back to the British when the Third Anglo-Dutch War ended and the name reverted back to New York. Until 1898, when the city was unified into the five boroughs, New York City exclusively referred to Manhattan, which is reflected in the address of New York, New York. The outer boroughs use their respective names, although the residents of Queens often put one of five neighborhood names instead (they are Long Island City, Flushing, Jamaica, Floral Park and Far Rockaway; which neighborhood residents choose depends on their zip code).

In North America (New) Plymouth and Providence had earlier shared the name between town and wider territory, though both later merged with neighbouring colonies.

It's also in accordance with a long British tradition in naming counties - Hertfordshire, Bedfordshire, Northamptonshire, Leicestershire, Nottinghamshire and of course Yorkshire, or the County of York, a style still common in 19th-century official usage and surviving into the 20th. Only Durham survives without the "-shire", as County Durham.

In France the practice may be older, though as in England (Sussex, Berkshire, Cornwall) other counties didn't carry city names. But of course it's far older, as in a recent discussion of Athens.

In the case of New York, it's worth realizing that 18th-century New York State's population was concentrated largely around the watershed of the Hudson River.

Much of what is today Western and Central "Upstate" was only lightly populated by Europeans for much of the 18th century. Nearly all of the cities and towns in Western New York, like Rochester, Buffalo, and Watertown, were not established and populated by Europeans until after 1800.

So when the British changed the name of their colony from "New Nederland" to "New York", the colony/state really was just New York City and it's hinterlands along the Hudson River and it's tributaries.