Part of the answer is also due to expected losses. The US and USSR relied upon the so-called Nuclear Trinity; Air, Land, and Marine, all of which are vulnerable to attack. Land based ICBM silos were possibly the highest priority targets for a nuclear first strike. Nuclear equipped bombers can be shot down and their airbases were also high priority targets, and similarly, nuclear equipped submarines and their bases were targets.

The two main philosophies of nuclear war both required excessive weapons to this effect. First Strike advocates said you needed to have enough warheads to completely destroy the enemy's nuclear arsenal in one attack. Defensive proponents said you needed to have enough nukes to survive such a strike and hit back. As was also mentioned, the exact numbers the enemy had was secret, so there was a tendency to overestimate to be safer.

That need to always have enough to obliterate your enemy's arsenal combined with the aforementioned increase in targets resulted in such massively inflated numbers.