My understanding is that the threat of mutually assured destruction made the use of nuclear threats between well-armed great powers and their core interests such as the US and NATO vs. the USSR and the Warsaw Pact not credible for fear of retribution worse than any profit that could be gained.
However, there were non-aligned countries which were not under any nuclear umbrella and were not part of any great power's core interests. Nuclear weapons had already been used in war by that point against Japan in World War II as a more efficient and effective means of delivering strategic bombing so as to compel the submission of a non-nuclear power to a nuclear-armed great power through the credible threat of annihilation (though if I understand it correctly, there are credible arguments to be made that in the specific case of WWII Japan the Soviet invasion of Manchuria had an equal or larger role than US strategic bombing in compelling the Empire of Japan to surrender because the invasion signaled the closure of the option of having the Soviets approach the Allies to bargain for Japan's conditional surrender, because what remained of the Imperial Japanese Army was mostly annihilated in Manchuria and thus would be unavailable to transport back to the home isles for the final defense, and because the invasion cut off the sources of what little food made it through the US Navy's blockade of the home isles thus bringing assuring starvation within days rather than months).
It would seem as though the example of Japan could have been interpreted by nuclear-armed great powers could leverage their arsenals to get countries outside of the core interests of the nuclear-armed powers (and thus outside of the credible threat of nuclear retaliation even if they got a treaty placing them under a nuclear umbrella).
Additionally, great powers during the Cold War were willing to use conventional strategic bombing on large scales against civilian targets to de-industrialize and compel non-nuclear powers to obey them as they had during World War II (strategic bombing during the Korean War, strategic bomding during the Vietnam War) Why did they not use nuclear weapons as more efficient means of achieving de-industrialization and compliance, as they had in the first and so far only uses during conflicts?
Finally, did policymakers in non-nuclear powers that weren't credibly covered by nuclear umbrellas believe that nuclear-armed great powers would use their nuclear arsenals against them if they went significantly against those great powers' interests? How did they pursue (foreign) policy in such a hostile environment?
Explicit use of "nuclear blackmail" was very rare in the Cold War, and tended not to work very well. So the US deliberately threatened to use nuclear weapons against China during the Taiwan Strait Crisis, for example. The result was China backing off... and starting its own nuclear weapons program. The only other clear case of it that I know of is in the Gulf War, where GHW Bush threatened Iraq with nuclear weapons should the latter use chemical weapons. It is not clear (and scholars have argued about this) whether this was a reason for Hussein not doing this (he did other things which made it clear that he didn't take such threats extremely seriously).
The US and Soviets on the whole were trying to gain favor with the Non-Aligned Movement nations, not lose it. There were exceptions to this (the Soviets and Yugoslavia and China, for example), but bullying a nation by threatening mass murder has generally not considered a great use of diplomacy over the years, and is a good way to drive them into the camp of your enemy. Nations that did feel existentially threatened in this way tended to either develop their own independent nuclear deterrents, or develop security relationships with another nuclear-armed power that extended their umbrellas.
On your general question, it's just worth dispelling the myth that Mutually Assured Destruction is the only reason that nuclear-armed states avoided nuclear conflict with one another. Deterrence in general (which is not the same thing as MAD) played a role in that, but for much of this history the balance of power was so far off that the US, for example, could easily have imagined using nuclear weapons in, say, Korea or Vietnam, without expecting retaliation in kind. That it did not use them points to other reasons not to use them — in those cases, fear of international disapproval, moral abhorrence, and fear of lack of effectiveness (they would not likely have led to the wars' conclusions) clearly played a role as well, as can be seen when we have opened the archives and looked at these deliberations.
While there were military officials in the US in the 1940s and 1950s who believed that nuclear weapons were "usable," these views were not popular among elected or diplomatic officials, who understood that global public opinion (and much of US domestic opinion) regarded nuclear weapons as being exceptional, fraught weapons. This growing sentiment about nuclear non-use — that nuclear weapons should not be used, as a normative position, and linked to frequently emotional responses to the atomic bomb, including those held by heads of state, such as Truman — is what is called "the nuclear taboo" by scholars. It was never an absolute thing, but as a global norm it seems to have generally increased over the course of the Cold War and 20th century (though whether it holds today, or will hold into the future, is a topic of debate).
Separate from the "taboo" model there is also a model that tries to say all of this is more rational ("the norm of non-use") and instead ascribes non-use to more cold-calculated conditions such as those I mentioned above: global public opinion (the US and USSR did not operate in political or economic vacuums; the US was and is in particular deeply dependent on foreign military bases for its military projection), and fears that unlike in World War II (if this is really what happened in World War II, which was and is debated) nuclear weapons use would not be decisive in these other conflicts (which were nothing like the conditions at the end of World War II). This could lower the bar for future use, among other possibilities.
On this, see Nina Tannenwald's Nuclear Taboo, which covers non-use generally in the US case.