Were there any communities or subcultures? How does it compare now?
Whoa, that's a big question! I can't answer it all, but I'll do my best. I'll limit my comments to the area that is now the United States, and begin with European contact with native American peoples, as I don't know much about pre-contact American sexual systems. My response here also focusses on European-influenced American cultures, as I can't speak to other cultural traditions of sexuality.
The first thing to realise is that, until the mid- to late- 19th century, there was no such thing as "homosexual." This is not to say that there weren't people having same sex intimacies; that's been going on for ages all around the world. What is important to remember, though, is that until relatively recently, a man's act of having sex with a man was seen not as indicative of an identity, but simply an act. Howard uses the useful term "homosex" to denote same-sex acts without invoking any cultural meaning that may or may not have been evident to the participants (Howard, p. xviii). In fact, the word "homosexual" was only invented in 1869 by Karl-Maria Kertbeny in German, and it was first used in English in 1892--just three years before Oscar Wilde had his trials (Bronski, p. xvii; Frankel, p. 7).
(Take this example: Today, some people prefer hamburgers whilst others like cheeseburgers. It's a personal preference, and there's no strong cheeseburger identity. This is kinda how people thought about sex partners before about the mid-1800s. Now let's say that Hamburger Lovers and Cheeseburger Lovers start to identify themselves by their preferred sandwich. Now, instead of just being someone who eats cheeseburgers, you're a Cheeseburgerite: it's part of how you identify. We're currently in a part of our sexual/burgery history where the object of our romantic/sexual/gastronomic desires are more than a preference, they're a way of life.)
Anyway, same-sex shenanigans were definitely going on in pre-20th century America. Homosex was typically discouraged, but in some surprising ways over the years.
"Sodomy" is a term that in the 19th century became heavily associated with anal intercourse between men, and it still has that connotation. But this was not always the case. Sodomy laws in the early British North American colonies were aimed more at heterosexual couples than at practices of homosex; anything that couldn't lead to reproduction was a capital offence (Bronski, p. 8 and p. 17). So much for masturbation.
Though you could be severely punished for violating a sodomy law, in practice things were sometimes more lenient. Nicholas Sension, for example, lived in Connecticut in the 17th century and was well known for asking men for sex and molesting his male servants. Although he was brought before the colonial court twice over a period of 20 years, he was not actually charged with anything until about 30 years after his first reprimand (Bronski, p. 9). In this case, homosex was explicitly illegal, but the law wasn't enforced as harshly as it could have been. The New England colonies had a complicated relationship with same-sex activity.
Moving to the late 18th and early 19th centuries, romantic friendships complicated the nature of what we now think of as gayness. Romantic friendships were really, really close relationships between same sex friends. These developed largely because of the homosocial world of the time: men went to school with men, worked with men, drank with men, etc (and likewise for women), so unsurprisingly they got really close to other men. Like, reeeeeally close, by our standards. For example, Daniel Webster had a close friend named James Hervey Bingham and wrote to him in 1804, "Yes, James, I must come; ...your little bed is just wide enough" (Bronski, p. 33). It's important to note that I'm NOT saying that Webster and Bingham necesarrily had or didn't have anything sexual going on--it was just totally acceptable for buddies to talk that way to each other, and sharing beds was common.
This is the background that laid the framework for nineteenth century sexual and social practices, some of which would eventually be labelled as gay. It is in this environment that we have Walt Whitman.
Whitman wrote erotically charged poetry celebrating the beauty of the working man and the power of male friendship. His "Calamus" poems were so homoerotic that, in this time before the invention of the word "homosexual," men who loved men were for a time called "calamites" (McKenna, p. 42). Whitman became really popular with a lot of the famous European characters in queer history, helping to unite the American and British strains of queerness that were beginning to solidify at this time. Whitman got letters from the likes of Oscar Wilde and Bram Stoker, and Edward Carpenter visited him in New Jersey (Bronski, p. 77; Rowbotham, p. 55). Clearly, Whitman's poetry spoke to something in the consciousness of people who would soon be identified as "gay."
That's about all I've got for right now. If you have more specific questions, I'd be happy to answer, but I'm kind of floundering with so much to say! Anyway, I highly recommend the Bronski book below. It's a very accessible introduction to American queer histories.
Sources: Bronski, Michael, A Queer History of the United States (Boston: Beacon Press, 2011)
Frankel, Nicholas, 'General Introduction', in The Picture of Dorian Gray: An Annotated, Uncensored Edition, ed. by Nicholas Frankel (Cambridge, MA: Belknap Press, 2011)
Howard, John, Men Like That: A Southern Queer History (Chicago: University of Chicago Press, 1999)
McKenna, Neil, The Secret Life of Oscar Wilde (London: Arrow Books, 2004)
Rowbotham, Sheila, Edward Carpenter: A Life of Liberty and Love (London: Verso, 2008)
EDIT: I found the McKenna page number and added it!