Crockett had been in New York in 1834 on a tour in support of his autobiography, A Narrative of the Life of David Crockett. He was taken around the city, attending the theater, meeting politicians, and touring the city's newspapers. The Five Points had already developed quite the negative reputation by the time Crockett visited the city, and he asked to be taken on a tour of the neighborhood.

Twenty years before Crockett visited the area, it was a fashionable enclave known as Paradise Square, with stately wooden houses built atop land that had once been the site of the old Collect Pond. The pond, once a favorite site for fishing and ice skating, had become polluted, and the city built a canal to drain it, filling the pond in with land from a nearby hill known as Bayard's Mount. However, the springs that fed the pond still ran, so the buildings started to sink. Moneyed New Yorkers soon left the area, and folks who couldn't afford other areas of the city moved to what became known as the Five Points, named after the intersection of five streets (and surrounding Paradise Square, which recalled the neighborhood's short-lived name).

The Five Points was often referred to in extremely negative terms, and Crockett is following suit in his narrative (he was also critiquing the 6th Ward gor being a large Democratic voting bloc). It was a major center of prostitution (the city had red light districts for years, but this one was seen as particularly immoral as some prostitutes slept with blacks), saloons and groceries sold copious amounts of beer and liquor (the latter served women, which was particularly worrisome to reformers), the streets were caked with excrement and garbage, and there were numerous gangs in the area who fought and rioted openly in the streets. All of this was shocking to the elites, and visitors flocked to the area to "do the slums" (Charles Dickens visited in 1842 and wrote a scathing account in his American Notes). Reformers such as the Five Points Mission had targeted the area beginning in the 1840s, and in the 1890s the city started condemning buildings, razing a large swath of the neighborhood for Mulberry (now Columbus) Park.

Hell's Kitchen, which stretched from the 30s (abutting the old Tenderloin district, where Penn Station is now) up to the 50s (where San Juan Hill once was, now the site of Lincoln Center), became another large slum district. The name is thought to have originated around the 1870s, and no one definitively knows where it came from, although it has long been common to use language describing heat and/or the underworld to describe particularly bad areas. It may have been the name of an area gang or a particularly notorious tenement (certain tenements had nicknames, like the House of Blazes). It could also have been bestowed by a city cop, as was the case with the Tenderloin district. When Crockett visited the city, the future site of Hell's Kitchen was still dotted with farmhouses, but by 1860s it was becoming industrialized, with the busy docks and surface railroad line attracting laborers, many of whom were new immigrants and blacks.