In New York City for most of the 19th and early 20th centuries, heat would have come from stoves. Initially wood stoves and then more commonly coal-burning, beginning with wealthier households around the 1820s and then becoming the norm across most housing. The crowded tenement-style housing you mention was prevalent in New York in the mid-to-late 19th century. Coal and sometimes kerosene stoves were used for cooking and heating in those rooms, and kerosene lamps were often used for lighting. (In crowded interior rooms with no windows, light and ventilation were a bigger concern than heat most of the year.) More expensive homes and fancy hotels featured indoor gas lightning and coal-powered central heat, where hot air was fed up from the basement through vents in the floor.

An 1865 survey of city housing found that stoves were prevalent in both middle and lower-class dwellings. In one neighborhood in present-day Soho, in the tenements "in which disease were found... stoves and kerosene are the main means of heating and lighting... gas rarely being used." While "private dwellings" also relied on stoves for heat but gas was the more common method of lighting.

In a tenement typical of another neighborhood along the East River waterfront, it describes a the first room:

... 14 X 12 feet with ceiling 8 feet high, having on one side two moderate-sized windows. The small fireplace is closed, and a stove exhausts rapidly the scanty atmospheric supply which finds its way into the apartment through crevices of the doors and windows. We observe that a pungent oder of coal-gas pervades the apartment.

In the adjoining room it mentions no heat source:

It has no other opening than the door of communication, and of course possesses no means whatever of efficient ventilation... [T]his is the dormitory of half a dozen persons. A sickening and stifling odor, most offensive to the unaccustomed senses pervades this apartment and poisons the atmosphere inhaled by the residents.

Therefore not all apartments and certainly not all rooms had heat sources. And even with a stove, people without the means to purchase coal or firewood would regularly spend winters without heat.

Some housing reforms went in later in the century, but even by the turn of the century, most people living in tenements and working-class housing couldn't afford central heat or new novelties like electric lighting. Stoves remained the norm, as seen in some of Jacob Riis's famous photos from the 1890s like this one of a tenement room and this one of a police station basement.

Even in upper-class dwellings, central heat that relied simply on rising hot air would not reach the whole home, so stoves or fireplaces with coal grates were still necessary in upper floors. Around the turn of the century steam heating systems began appearing in commercial buildings. In the early 20th century steam heat was available in some newer housing projects aimed at the working class, like the First Avenue Estate on 64th St, built by City & Suburban Homes. But for the first few decades it remained mostly the domain of new apartment buildings targeting middle and upper-middle class renters.

Sources: Gotham by Edwin Burrows and Mike Wallace, and Greater Gotham by Mike Wallace.