A few reasons:
Geographical determinism: Long Island and Staten Island protect the harbor (the southernmost tip of Manhattan was easily defended in case of attack), the deep water port produces less ice than shallower ports (i.e., Boston), and the completion of the Erie Canal in 1825 provided a connection to the Midwest and the Great Lakes.
European politics: the British, resentful for having lost their colonies, preferred not to trade with revolutionary cities (i.e., Boston). Thus, after the War of 1812, New York, which had significantly more Loyalists during the Revolution, became a dumping ground for European goods.
A lot of capital-intensive industries that relied on imported raw materials (i.e. sugar refining and garments) ended up there for reasons 1 and 2. And since the first few garment/sugar factories arose in NYC, this attracted more sugar/garment workers to the city, which in turn attracted more industrialists looking to build sugar/garment factories, etc. In a phrase, "cumulative causation" due to high transportation costs, a need for specialized employees that were already based in the city, and relatively minor comparative advantages that compounded over time.
My source is what I can remember from Edward Glaeser's article "Urban Colossus" (Warning, PDF:http://scholar.harvard.edu/files/glaeser/files/hier2073.pdf)