What you're thinking of is a "Shibboleth," a linguistic quirk distinguishing a particular class or group of people.

Now, differences in varieties of American English aren't as pronounced as differences of varieties older languages, which is why Shibboleth's like "House-ton Street" stand out. Plus, modern New York City is a magnet for people from all over the world, sometimes it's hard to understand if that guy with the funny accent has been here ten days or ten years unless you hear him name an address in SoHo.

It's much easier to figure out where someone is from if they're speaking a language like Italian, or French, where every region has their distinct accent and dialect. A well-trained ear can even pinpoint a person's hometown to some degree of accuracy based on their accent. However in in both modern France and Italy, many people perceive their "Standard" language as a "Prestige Dialect" and will almost always use standard Italian when communicating with strangers. Many middle-and-upper class speakers are further able to tone down their native accent in favor of standard pronunciation, to the point that their native accent is imperceptible.

Of course, in the past (especially before the advent of television and radio) it was very difficult for a person to lose their native accent even if they learned the standard language. The jury is out as to how strong Napoleon's Corsican accent was, for example. Another example is the 1100-year-history of the Republic of Venice, where a citizen of the Republic traveling from Padua or Treviso to Venice on business would be immediately recognizable due to subtle differences in his speech. A simple Shibboleth? When inquiring what a friend or associate was up to the previous evening, a Trevisan would ask, "Cossa gatu fatto?" whereas a Venetian would ask, "Cossa ti ga fatto?" The accent is near identical, but the Venetian inverts the middle syllables and adds a pause.

As for cities referred to as "The City," I don't know of any evidence with regards to Rome, but the name "Istambul" comes from the Greek phrase, "Isimpolin," meaning "to the city." This name, very nearly meaning "The City," replaced the city's previous name, "Constantinople."