I always had this idea that it was men with one hand inside their coats having deep philosophical and political debates. Or was it just a bunch of ye olde bros yelling about darts and body odor.
It depended on what tavern you were visiting.
From Old Taverns of New York:
"Before any newspapers appeared, the tavern was a very important institution in the community. It was the medium of all news both political and social, the one place where people of all kinds met to exchange views on every subject of interest to the general public. In this way, it exercised an influence second only to the church....
"They were frequented by all classes and the influence of each one of them on the community depended entirely on the character of those who patronized it. The merchants, the politicians and the men of letters each had their places of rendezvous...
"Nothing exists at the present day in any way resembling an old tavern of the first class in colonial times. It was the place for political discussion, for social clubs and for meetings of all kinds. Every one went to the tavern and from no other source could a person gain so much knowledge of public affairs."
Source: Old Taverns of New York, pp. xv-xvi, published 1915
And from The Old Boston Post Road:
"The name tavern had a different signification in those days from what it does now, when we associate with it one idea only, that of selling liquor. In the earliest days of the colonies, each town was empowered to select a good responsible person for the keeping of an ordinary for the entertainment of travellers; and the colonial laws were very strict in regard to the conduct of the ordinary, even regulating the strength and price of the beer that was sold, and making it obligatory upon the landlord in some cases to report to the authorities any sojourner who stayed for more than a few days.
"The tavern was the most important place in town; it was the gathering place to learn the news, it was the business exchange for the neighborhood, the place where bargains were made and prices learned and quoted. It was at once the town-hall and assembly room, the court-house and the show-room, the hotel and the exchange. Itinerant actors and showmen gave their exhibitions in its public room, strange animals and curiosities were displayed at the tavern; here were the bulletin boards containing the lists of jurors, the notices of vendues, legal notices, rewards for runaway slaves or servants, for lost animals or other property, and the farmer's advertisements of what he had to sell or of what he wanted to buy.
"The taverns of New England were famous at the beginning of the nineteenth century for their neatness, cleanliness, and comfort, and for the excellence of the food; and this at a time, too, when all foreign travellers were commenting upon the carelessness, the crowded conditions, the filthiness, and discomfort of the taverns on the roads to the rapidly developing West."
Source: The Old Boston Post Road, pp. 31-32, published 1913
These two books give descriptions of specific colonial era taverns in New York, Boston, and points in between.
EDIT:
Further reading:
The Colonial Tavern; A glimpse of New England town life in the seventeenth and eighteenth centuries, published 1897
Home Life In Colonial Days, published 1898
Stage-Coach and Tavern Days, published 1900
The Turnpikes of New England and Evolution of the Same Through England, Virginia, and Maryland, published 1919