I have made some progress and found some businesses that are interesting, but I suspect there are more bars, dancehalls, etc. Of note, The Metropolitan Hotel, where 'professor' Jerry Thomas, inventor of the Tom & Jerry cocktail, was the bartender. Also, it is the home of Stanwix Hall... the bar where William Poole (also known as Bill the Butcher from Gangs of NY) was shot.
There were other bars on this street and I want to find out what they were! Any suggestions would be appreciated.
Here's a link to an 1865 City Directory, with every occurrence of the keyword "saloon" highlighted. I also did a keyword search for "bar" but there were no search results (except for iron bar manufacturing).
https://books.google.com/books?id=hY4tAAAAYAAJ&printsec=frontcover&dq=city+directory&lr=&as_brr=1#v=onepage&q=saloon&f=false
Here's a map of Broadway between Spring and Houston from 1857-1862, showing all the address numbers. It may be helpful with the corner addresses that you mentioned didn't have numbers in the city directories. https://digitalcollections.nypl.org/items/510d47e0-bf4a-a3d9-e040-e00a18064a99