Why was Andropov and the Kremlin elite in general so paranoid in 1983 as to almost cause a nuclear war? Were they senile? I mean I get it, they went through WW2 and Barbarossa, but still. And why at that point of all times? Why were they so scared of Reagan? Why did they believe a first strike was coming? Did they not realize they would always be able to launch a massive retaliation to destroy the US, even by submarines alone? Surely they knew that. By the 1970s and 1980s there was easily MAD. Why were they scared of the Pershings? They had hundreds of SS20s as well! What else gave them their opinion that the US was planning a strike? Were they also fed false information by the Stasi or others?
The specific fear that the Soviet elite had was that the US, with its vast arsenal of very fast and very accurate weapons that could be placed very close to Soviet borders, could execute a "decapitation attack." That is, they believed that the US believed that it might be possible for the US to destroy their means of retaliation, first by destroying their command and control systems (e.g., killing off the Politburo, the top generals, and the means by which a nuclear "firing signal" might be sent), and then attacking the weapons themselves ("counterforce"). They also worried that the US might have the ability to detect the location of Soviet submarines and thus blunt their capability as "second strike" devices.
Is this paranoid? The US, at the very same time, worried that the Soviets were developing the exact same capabilities. Much of Reagan's defense posture was in fact based on the idea that the Soviets had pulled vastly ahead of the US and that there were now potential "windows of vulnerability" where the US could imagine how it would attack itself and render its ability to respond either impotent or at the very least blunted. So if it was paranoia, it was a paranoia felt by both sides: advances in the 1970s had, in the eyes of many, destabilized the balance of power.
On Pershings vs. SS-20s — Pershings were feared as weapons that could destroy command and control. SS-20s did not serve the same aims (they could not target Washington DC and NORAD, for example, but the Pershings were feared to be able to target Moscow). So there is an imbalance there, an imbalance created by the fact that the US had nukes on the territory of allies on the borders of the USSR (like West Germany) and the Soviets did not have the reverse (the one time they tried to do this, in Cuba, led to another war scare).
The USSR brass genuinely believed that the USA was contemplating a paralyzing first strike, one that might end up with the destruction of Western Europe, and maybe some losses in the USA, but in their eyes they thought the US leadership might be anti-Soviet enough to think it worthwhile, if it also led to the destruction of the USSR. Though there had been, in the 1950s and 1960s, advocates of preemptive war in the USA, no President ever took such a thing very seriously, and in retrospect this was a totally incorrect assessment by the Soviets, reflecting a deep misunderstanding about the priorities of American Cold War Presidents (none of whom would have been willing to trade US cities, or Western Europe, for the end of the USSR). Of course the US had similar misconceptions about the designs of the Soviet Premiers, we now also know.
With this framework in mind, the KGB and Stasi sought out confirmation of their worst fears in any piece of data that they saw. Statisticians call this "chasing noise": when you are willing to take in any evidence for a thesis as "positive" evidence, no matter how small or inferential, and you do not take into account any evidence against it (because you think it is part of a conspiracy, or whatever), then you find endless confirmation for your thesis wherever you look. This is the cognitive error that underlies all conspiracy theories, but it also can enter into intelligence assessments as well, especially if there are political rewards for giving the bosses what they hope to hear (which may have been the case with the Stasi). (And again, this is not unique to the USSR: consider how US intelligence on Iraq's WMD program went so wrong.)
Even if Reagan had not made quasi-apocalyptic statements (that to a Soviet mind confirmed their worst fears about the influence of religion on these matters), even if Reagan had not campaigned on and then went through with the idea that the US was vastly behind the USSR and needed a vast upgrade in nuclear weapons systems, even if Reagan had not joked about starting nuclear war against the "Evil Empire," even if they had not based new, fast, accurate Pershing II missiles in West Germany, even if they did not plan massive war game exercises like Able Archer right near Soviet borders, even if they didn't violate Soviet airspace repeatedly to "test" their reaction capabilities (which led to the shoot down of KAL007), etc. etc., you could imagine the Soviet Union, once it fell into this mindset, being very uneasy about US intentions and goals. But certainly with all of those things, it is easy to see how this mindset flourished under these conditions, only to be undone by better understanding (under Gorbachev) of the US state of mind.
Because the control over the use of nuclear weapons is very tightly controlled at the top of governments, the game of guessing how they would be used becomes a very personal, even psychological one. It's relatively easy to talk in broad terms of national policy and philosophy (though even that can lend to perilous misunderstandings and generalizations), but if you're trying to characterize an individual President or Premier you run into all sorts of idiosyncrasies, personal beliefs, and even questions of psychological fitness. So it is easy, if you are in that mindset, to assume the worst case scenario. And when the consequences of being wrong or right are so high, as they are with nuclear war, then the strategists and intelligence analysts have easily found it defensible to assume the worst until you have confirmation of the alternative.
Hoffman's The Dead Hand is a great resource for characterizing the US and USSR points of view in the early 1980s, esp. in 1983.