I understand that the Cold War was a political struggle between the Western Bloc and the Communist Soviets/Eastern Bloc and the US and Russia directly fought, but how can it be "cold" when more than 3,000,000 people were killed in the Vietnam War alone?
The three wars I'm referring to are the Korean War (~2.5m Dead), Vietnam War (~3m+ dead), and Soviet-Afghan war (~1.5m dead).
Does it really just come down to the fact that no one dropped the bomb on the other?
It's useful to look at the Cold War in a broader context.
Some historians have compared the Cold War to a similar period of rivalry and competition between Great Britain and Russia for dominance of Central Asia in the 19th Century. (For a fascinating and cracking read on this, I cannot recommend enough Peter Hopkirk's the Great Game.) The difference between this rivalry, and the Cold War, is that Britain and Russia came to blows over this (among many other issues) in the Crimean War.
In the past, great powers' whose interests ran at crosspurposes typically got down to business and mixed it up. The Cold War was different because at no point did American and Russian personnel officially engage one another. (Pilots in the Korean and Vietnam Wars, however, on both sides knew this was essentially a legal fiction.) The Cold War was a series of many proxy battles, spanning the globe, where each power sought to use its respective proxy in opposition to the other. Now, this is nowhere close to new in global affairs: one need only look to Thucydides and Xenophon to see that Sparta and Athens were playing this game long ago. The difference, of course, is that Sparta and Athens fought it out, just as Britain and Russia had. I am not an expert in Cold War theory and practice, but this is a crucial, and I would argue, non-rhetorical, difference.
The introduction of the atomic bomb did change things - but whether its effect was purely psychological or not is a question beyond the one you're asking.