I'm talking about the Upper East/West Sides in particular (as a native of that part of the city). I just came from a thread on /r/nyc documenting the history of New York's public transportation. It featured a photo of 86th Street and Second Avenue without a building to be seen-- there was nothing but a paved street and grass on both sides. I know that until the early 1900s Lower Manhattan was by far the most urbanized part of the island. What are the reasons for Upper Manhattan's rapid development and rise in population density?
The 1904 opening of the IRT subway (followed quickly by other lines) made travel much easier to the downtown business district, spurring development in outlying areas.