In the 17 and 1800s, how did gay men find other gay men for sex?

(I'm focusing here on Britain. Things probably worked differently in different parts of the world.)

tl;dr Carefully, and with a reliance on subcultural signs.

The first thing we need to do is to establish that, throughout the 1700s-1800s, it was illegal for men to have sex with men. But just because a law is on the books doesn't mean it's enforced. The level of intensity with which men having sex with men was actually policed varied across time and place, and is the subject of a whole contentious body of social history scholarship. In general, though, you did not want to get caught in bed with another man. After 1885--the implementation of the Labouchere Amendment--you also didn't want to get caught even canoodling with another man, because "gross indecency" between men, even in private, was criminalized.

So throughout this period there was a need to keep sexual contact between men hidden from the authorities, but of course there was also a need for men seeking sex with men to be able to identify each other. Such concerns would have existed prior to the 18th and 19th centuries, but during this time there was a major population shift from countryside to city (due to the Industrial Revolution), meaning that more and more men were living in increasingly anonymous cities with lots of potential sexual partners, but also lots of opportunities to get caught. So how do you find partners in a big city without attracting the authorities' notice? Subcultures! A subcultural system of signs developed over the course of this period--and beyond--that allowed men "in the know" to identify each other while metaphorically flying under the gaydar.

In big, anonymous cities like London and Manchester, cruising (going to public places with the intention of picking up a sexual partner) was a good way to find each other. Over time, there developed a whole cruising geography of cities. Certain parks, public toilets, train stations, etc. would become subculturally known as good places to find partners. Some professions were also known for having high numbers of men who liked to have sex with men--or who could be persuaded to do so for the right price. Enlisted men in the military were particularly known for their willingness to have sex with men for cash.

Actually identifying each other could be a visual or auditory process. In the late 1800s, middle-class and elite men started wearing green carnations in their buttonholes as a way to identify each other, for instance. There were also certain code words or patterns of speech that would tip a man off to the interest of a potential partner. By the mid-20th century, this had developed into polari, a whole dialect that men could use to identify each other. You can watch a short film in polari here on YouTube; a helpful commenter has given a summary in standard English, so this is one time that you definitely do want to read the YouTube comments. The Houlbrook article listed below gets at the erotics of smoking; as today, asking for a light or a cigarette could be a convenient way to make eyes at an attractive man.

Perhaps surprisingly, an early version of what we know today as the gay bar existed in the late 1700s. These establishments were called "molly houses," as "molly" was an 18th-century slang term for an effeminate man who had sex with men. The patrons of molly houses would have been a fairly close-knit group of regulars who called each other by typically female names and often wore dresses to the molly house (but likely would have worn breeches or trousers and used male names outside of this context).