The shortest answer to your question is that nuclear forces are a very large hammer, and sometimes the political goals of the Soviet Union (or any other country) require something less. Nuclear forces are a bit of an "all or nothing" guarantor of security, and the problem of deterring conventional aggression with nuclear arms has (fortunately!) never been fully settled. Conventional forces are oft seen as a way to deter "medium intensity" wars, whereas if you only have the nuclear deterrent, your opponent might gamble that their goals are limited enough that you wouldn't risk nuclear annihilation over them. (Example: The Soviets would risk nuclear war for Moscow, but for East Germany? Maybe not.) This was not always assumed to be true, as I'll discuss more in this answer, but this idea that there are different types of military force with different uses will serve us well as we tackle this question. One case of this type/use relationship which holds true for all of the time periods I intend to discuss is quite simple: nuclear weapons cannot hold territory.

The question-behind-the-question though, the real heart of the matter, is "what is a sufficient deterrent?" It's an enduring question in modern international relations theory, and it was debated in the US and the USSR all throughout the Cold War. I will be invoking a lot of argumentation here from Keir Lieber and Daryl Press' fairly recent Myth of the Nuclear Revolution: Power Politics in the Atomic Age. For good summaries of nuclear deterrence in its various historical contexts Lawrence Freedman's The Evolution of Nuclear Strategy is a fantastic introduction, often used in college courses. Accurate specialist literature on Soviet concepts of deterrence is harder to come by, and it's honestly a topic I'm still learning about. Most of the writing we have access to is unclassified (or rarely, declassified) American assessments from the Cold War; these are flawed evidence for a number of reasons. The Soviets did publicly discuss matters of strategy in the nuclear age - some of these works have been translated into English, like Sokolovsky's Military Strategy, though many salient journal articles are still behind both language and access barriers. Nuclear warmaking and ideas of deterrence, though, were far more rarely discussed in the open. For our purposes, the sections on Soviet deterrence concepts in Freedman are probably good enough.

There are four basic "types" of nuclear deterrent which we can arrange by increasing level of investment, complexity and readiness. The leanest deterrent is often called an "existential "deterrent, where the simple fact that a state has nuclear weapons is sufficient to deter enemy aggression. Existential deterrent theories hold that it does not matter how many nuclear weapons you have, or what their technical characteristics are, or even (in extreme versions of the view) whether they are kept assembled. The idea is that once a state has a handful of nuclear weapons, the risk of nuclear retaliation goes from zero to somewhere above zero, and no rational leader would accept this risk no matter how slight. De facto this was the state of the Soviet deterrent from their first nuclear weapon in 1949 until about 1956. They had nuclear weapons, but in practice they had no reliable way of delivering them to the United States, only a handful of bombers whose range was not sufficient for anything more than a one-way suicide trip. Therefore, this time period obviously still required a large conventional military.

The next step on the ladder is often called "minimum deterrence". This concept is concerned with the smallest nuclear force which can plausibly deliver a retaliation against aggression. Where existential deterrents are satisfied with the possibility of retaliation, minimum deterrence advocates believe that the retaliation has to have some real plausibility of succeeding. In the Soviet case this describes the period from about 1956 to 1962. In this time period, the Soviet long-range bomber and ICBM forces could effectively strike the United States. What differentiates a minimum deterrent from the next spot on the ladder is that minimum deterrents are not designed with a high degree of survivability. A nuclear first strike by the United States had fairly good odds of catching them with their pants down. In this time period the Soviets did attempt some force economization under Khruschev, who was a big believer in both nuclear weapons and missile technology in general. But the conventional arm of the Soviet land forces remained strong in this time. Why? We can partly explain this by saying that the Soviet defense complex and the army had a lot of political power and pushed back on Khruschev's reforms. However, the reality is that the Soviets did not entirely believe in the ability of nuclear forces to deter conventional aggression, and they did not really believe that NATO was likely to open hostilities with a nuclear barrage. The conventional forces were required to defend against NATO conventional forces. As described by Vitaly Tsygichko, a strategic analyst for the Soviet General Staff from 1962 until 1985, the Soviets spent the entire Cold War believing that neither side wanted to start the war with battlefield nuclear weapons, but would likely resort to them fairly quickly in the face of tactical failures. Hence, in the initial period of any war, the Soviets needed a conventional force because they expected NATO to attack with a conventional force. It sounds a little chicken-and-egg, and maybe it was.

From 1962 onward, the Soviets held what we would call an "assured retaliation deterrent" in the form of longer-ranged submarine-launched missiles, ICBMs in silos which required far less warning to launch and could theoretically be launched as soon as the Soviets saw American missiles on radar, and later on the adoption of mobile ICBMs on trucks which could be hidden and dispersed in a crisis. The assured retaliation theory of deterrence says that a state is secure when the enemy cannot realistically expect to destroy its nuclear forces in a first strike, no matter how well the circumstances go for them. This tends to bleed over into the fourth and most expansive model of deterrence, the infamous assured destruction model. By the mid-60s, the Soviets and the Americans were indeed in a state of mutual assured destruction. Not only could each side expect to launch a number of nuclear weapons after the enemy's first strike, each side expected that their second-strike retaliation would be large enough to completely destroy the other nation. Under such conditions, one could imagine that paradoxically a conventional war was more likely, since each side might feel more restrained in its initial resort to nuclear weapons. And of course the question of each side's nuclear use threshold remained. If NATO invaded East Germany with conventional forces, and the Soviets were about to suffer operational defeat in East Germany, would that be enough to launch nuclear weapons? Or instead would the conflict stay conventional so long as NATO did not cross over into Poland as well, for example? Hence the need for very strong conventional forces, to prepare for all eventualities and hopefully deter NATO from using conventional forces to test the lower bounds of Soviet nuclear use.