The US military and most defense scientists essentially responded to the nuclear winter hypothesis by saying, "there are many uncertainties and unknowns, and this is very hard to calculate, so I'm not going to take it seriously." This was not universal; in fact, some defense scientists and laboratories did do their own simulations with their own guesses at the unknown variables, and came up with their own assessments (some of which validated some version of the hypothesis, or at least put thresholds on what the true values of the variables might be). But the general response by the political establishment was to ignore it — they saw the nuclear winter hypothesis as being essentially unknown, and to take it truly seriously would mean a radical rethinking of US defense posture, and they weren't willing or able to do that.

The similarity to climate science (aside from the fact that nuclear winter forecasting actually led to some climate forecasting models) is in this kind of selective uncertainty bias. There is always uncertainty in science, especially for large-scale hypothetical situation modeling (as with nuclear winter and climate change), and if you are inclined to not want to take the model seriously you can say, "well, there's a lot of unknowns there, and you're making a worst-case assumption about what might happen." The same people might turn around and make worst-case assumptions about other areas of uncertainty (like how big the Soviet arsenal might be), so it's not like worst-case assumptions are universally ruled out by them (or anyone; there is a lot of subjective judgment involved in these kinds of things on all sides).

It's worth noting that some of the doubters were people who were otherwise in favor of disarmament or arms control — they just weren't sure they wanted to put all of their eggs in that particular basket, given the uncertainties. In some ways it didn't help that Carl Sagan was associated with, because for many of such people (rightly or wrongly) they had started to regard him as more of a popularizer/politicizer than active scientist. The fact that the theory made its first real public appearance in Parade magazine didn't help that. Sagan's interest in "getting out the message" may have backfired in this instance, by undermining the apparent legitimacy of the work.

There were some who believed that the nuclear winter hypothesis was some kind of Soviet hoax or propaganda — this is provably untrue for the original Americans who engaged in the work (Sagan et al.) but the Soviets did support basically any idea that seemed to call for reduced American nuclear deployments, so you can see why that was popular with a certain set.

Anyway, there is a really good book about all of this that, if you're curious about it more, I recommend highly: Lawrence Badash, A Nuclear Winter's Tale: Science and Politics in the 1980s (MIT Press, 2009). He covers the whole episode very carefully and very impartially. On the ways in which organizations can "see" what they want to see, even when they ostensibly are highly technical, one might also look at Lynn Eden's Whole World on Fire: Organizations, Knowledge, and Nuclear Weapons Devastation (Cornell University Press, 2004), which looks at how the US defense establishment consistently underestimated the fire effects of nuclear weapons because they were hard to calculate relative to blast effects.

As for Mutually Assured Destruction — there were a lot of variations of thinking about US nuclear strategy over the course of the long Cold War. MAD only really takes form in the 1960s; prior to that, things were much more lop-sided in favor of the US. True parity was not reached until the 1970s-1980s, but by the 1960s the USSR could do a lot of damage if it had to. Prior to the mid-1960s there were still occasional voices who thought that the USA ought to preemptively attack the USSR; that idea mostly gets relegated to crank-dom by the time the USSR has a functional ICBM and later SLBM fleet. The question of whether nuclear weapons use necessarily need "escalate" to full on mutual destruction is a broader problem that was debated by academics and think-tankers (e.g. Herman Kahn) through the 1960s-1980s but largely settled into an area of "it would be hard to avoid escalation in a real-world situation." It is of note that many top-level politicians from the Cold War thought such "academic" theorizing was pointless, that they would not rely on slivers of probabilities to make such major decisions. On the evolution of nuclear strategy, see the appropriately-named Evolution of Nuclear Strategy by Lawrence Freedman, which is the canonical text.