It seems like there are two parts to the question: Were the '80s and '90s less progressive and accepting of LGBT people than the '60s and '70s? And if they weren't, why do the movies that you've been watching give the opposite impression?

The first half isn't too difficult to answer. There are reams of evidence that the '60s and '70s, while more open to LGBT rights than the decades prior, were much less progressive than later decades, and that society has evolved on these issues in a more or less linear fashion -- going from less accepting to more accepting.

Let's start with the best way of determining what the public at large thinks: polls. The first poll asking about gay people (I'm saying "gay people" or "homosexuality" because LGBT as a catchall is a very recent acronym, and the B and T often weren't part of the conversation around these issues in the decades we're discussing) was in 1965, and it found 70 percent of Americans thought "homosexuals" were "harmful" to American life.

Then there isn't a lot of polling for another decade. But there is a lot in 1977, and it's interesting for a couple of reasons.

First, why 1977? Because in 1977, Dade County, Florida, decided to prohibit discrimination based on sexual orientation. It sparked a massive conservative backlash -- which might seem like it would support your hypothesis. But it's important to remember that the backlash happened because there was an actual widespread legal movement to stop discrimination against gay people for the first time. It's not that in the 1960s nondiscrimination ordinances passed without protest; it was that no one even proposed nondiscrimination ordinances.. Progress and backlash move in concert -- the backlash wouldn't have happened without the progress.

Anyway, so, in 1977, polls suddenly start asking a lot of questions about homosexuality. Gallup, for example, started asking in its polls if gay people should be allowed to do certain jobs. (The idea of this question existing at all should give you an idea of how distant attitudes from that era are to most people now.) Gallup also asked if homosexuality should be legal and found the country evenly split, 43/43.

If your hypothesis were correct, we'd see more progressive attitudes in the late 70s, then a regression through the 80s and 90s, and then the numbers would begin to rise. But for all of these questions, there's a steady climb toward more acceptance.

Still, it's not like there's a ton of polling about homosexuality, and when these questions were asked it's usually because some bigger controversy is in the news. So what else can we look at?

Well, what about those anti-discrimination laws? In the 1960s, there weren't any. In fact, the situation was the opposite: in the "lavender scare," homosexuality was (supposedly) rooted out of the federal government because gay men and women were thought to be a security risk. This began in the 1940s and continued into the 1960s, the period under discussion. It wasn't until after the Stonewall uprising in 1969 that the gay rights movement picked up serious national momentum and anti-discrimination laws and ordinances came under discussion. The first organization to fight for legal rights for gay people, Lambda Legal, was established in 1973. The first federal gay rights bill was introduced two years later, but died quietly in committee in Congress. (Edited to correct “died on the floor” — it never made it to a full vote.)

There's some local action through the 1970s. What happens in the 1980s? Are those ordinances repealed? Is there retrenchment? No, they keep passing. The Democratic Party puts gay rights in their platform for the first time in 1980. Wisconsin (!) was the first state to pass an antidiscrimiation law in 1982. The 1980s weren't a decade of remarkable progress -- among other things, the Supreme Court upheld a law that criminalized homosexual activity, there was a lot of public hate and bigotry sparked by the AIDS crisis -- but the numbers suggest that, over time, America was generally softening on the idea of homosexuality.

Then we hit the 1990s. And compared to basically any decade that went before, the 1990s look like a damn Pride parade. Over the course of the decade, opinions about gay rights "changed dramatically" (in a positive direction). Congress almost passed a national non-discrimination law for the first time. Don't Ask Don't Tell is remembered as a virulently anti-gay policy today, but at the time it was actually a compromise where one of the alternatives was continuing to ban gay people from the military. For the first time, polls start asking about same-sex marriage — it's extremely unpopular, but fact that they're asking about it at all, when 20 years prior (edited: not “later,” d’oh) the question was "should gay people be able to do whatever jobs they want?" is itself an evolution.

So that's the trajectory of LGBT rights and acceptance: there are cycles of progress and backlash, but overall, the rise has been linear and quite rapid. Which brings us to the second part of the question: why don't the movies make it seem that way? That is actually a super interesting question and less straightforward (pun not fully intended) and I'll generally leave that question to scholars of film history. But I'll note that the movies you have been watching are a very, very small sample size of contemporaneous media. If you look at TV, for example, you see a growing number of gay characters on mainstream shows in the 1980s.

And, of course, at the end of the period you're asking about, you get the debut of Will and Grace — if we're talking about the representation of homosexuality in pop culture, you can't not talk about Will and Grace, which literally changed people's minds about gay people by making them feel like they did have a gay friend -- that nice and funny man in the teevee show.

As my background is statistical, I'd like to approach your question from the standpoint of confounds and data methodology. Shaft, Dog Day Afternoon, and Midnight Cowboy are gritty, and that wasn't unusual. Shaft is the standard for blaxploitation (it and Blackula are the only ones most people can name, and it's the only one most people know anything about) and Dog Day Afternoon and Midnight Cowboy are standard New Hollywood fare. Shaft and Dog Day Afternoon are good examples of neo-noir, and Midnight Cowboy is adjacent to the genre. In other works, they are examples of an era in Hollywood when the focus was on countercultures and marginal people and groups, typically of a type found in major metropolitan settings rather than rural or rust belt, and meant to make the viewer somewhat ambivalent at best. This means that films were tending to be set in places one would expect to find gays, such as the East Village, and on people the audience might not approve of. The three 1967 movies that kicked off the movement were about murderous bank robbers, a noef (Hebrew for "adulterer," with Judaism's somewhat particular definition), and a prisoner in a chain gang!

This all changed with the runaway success of Jaws, Star Wars, Raiders of the Lost Arc, and First Blood (as well as a number of behind-the-scenes incidents that soured much of Hollywood on the style of film-making and direction). All of a sudden, films weren't about living on the edge in Avenue D, but instead fighting terrorists/thieves in Nakatomi Plaza. Similarly, while History of the World Pt. 1 may have promise Jews! In! Space!, you certainly weren't going to meet any in The Empire Strikes back (although you do see a Hasid davening in Ghostbusters, a nice acknowledgement of the setting). As a parallel trend, the number of Latins and Native Americans depicted on screen had absolutely collapsed by the end of the 1970's, not because Americans had turned against those groups, but because they were largely only depicted in westerns and that genre died.

​

This is somewhat similar to how you might see rising numbers of rural-associated populations in a country's census as it gets its shit together because it's able to actually count them or even just takes a greater interest in them. Similarly, our knowledge of what life was like in Talmudic times is heavily constrained by what topics the (proto-)rabbis of the time thought needed discussion (either due to importance to them or intellectual complexity). I'm not saying that gays didn't have a diminished place in public life, as AIDS for example did a huge number on their population, but that major studio film are not necessarily following population trends in depictions over time.