Firstly, there were waves of cruise missile development. The early generations were essentially wholly inferior to ICBMs of the time. The first ICBMs (e.g. Atlas and R-7 with non-storable liquid propellants) had truly global ranges and sufficient accuracy to make them worthwhile weapons of war. Being able to station ICBMs in the most protected inland areas made them more useful in an attack.

Through the 1960s there was a flat out race to build ICBM (and then SLBM) capacity by any means necessary, with a variety of different launch platforms and designs employed in the ICBM fleets of both the US and the Soviet Union. Through that time the forces ramped up from a handful of launchers to hundreds. One important thing to note here especially is that ICBMs had become capable of being kept on constant alert and housed in hardened silos fired from protected underground bunkers. In 1965, for example, an ICBM could be fired on a moment's notice any time of day or night and hit its target in under an hour. That was a capability that cruise missiles at the time weren't capable of matching. Nor was there enough extra money to build and deploy an intercontinental cruise missile system. In the US the "nuclear triad" had already been cemented: with submarines, SAC, and land based ICBMs. And that system was enormously expensive already. Additionally, there wasn't a large window of time here to shoehorn in another megaweapons development program.

Starting in the late 1960s the nuclear arms race went through a very significant phase change or inflection point. Targeting accuracy improved greatly, nuclear weapons became much more efficient/compact, and two developments dramatically changed the entire landscape: multiple independently targetable reentry vehicles (MIRVs) and anti-ballistic missile (ABM) defenses.

The Minuteman III solid fueled ICBM first deployed in 1970 had the ability to launch 3 separate thermonuclear warheads that would be deployed during the cruise phase of launch by a spacecraft bus to each hit individual targets. This meant that every single Minuteman III ICBM was worth 3 other ICBMs. Additionally, the Minuteman III system was capable of much greater accuracy. The Titan II could reliably hit the downtown core of a large city with a multi-megaton nuke that would nevertheless be certain of destroying the city. The Minuteman III could hit a particular city block with a 170 kiloton warhead that would nevertheless be certain of destroying the city. This double efficiency of being able to make more efficient use of fissile material to more accurately deploy destructive damage against targets was a massive increase in the capabilities of nuclear forces.

On the other side of that, the advent of digital electronic computing made ABMs possible. In the 1960s nobody was looking for finesse, they were just looking for the ability to maneuver another nuclear warhead near enough to an incoming warhead so they could blow both to smithereens. A sub 10 kiloton high (or exo) atmospheric nuclear blast is vastly more survivable than a precisely set several hundred kiloton to several megaton blast in the heart of a population center (or strategic asset). So the ABMs that were developed at the time were actually fairly effective.

However, these dual developments kicked the nuclear arms race into overdrive. ICBM deployments went from hundreds to thousands. ABM development grew and grew and grew. With MIRVing technologies it was immanently possible to add decoys, chaff, and other "penetration aids" to the missile which would overwhelm the ABM defenses. In a world with heavy ABM deployments it became necessary to build huge numbers of ICBMs because one had to imagine many of the warheads would not make it to their targets. And that in turn drove even more rampant ABM deployments and massive expenditures on ABM technology developments.

Now, compared to a heavily MIRVed ICBM/SLBM a cruise missile doesn't seem to hold much advantage, since you can't reasonably MIRV cruise missiles. However, in a world filled with ABM systems cruise missiles start to make sense, because they can more effectively evade ABMs.

However, just as this MIRV/ABM really started kicking into high gear it was brought up short by none other than international diplomacy. Fortunately, people saw where things were going and enough sanity prevailed for the US and USSR to come to the table and make agreements to keep the nuclear arms race from spiraling out of control. This led to the SALT talks and the ABM treaty, which limited ABM deployments and effectively put the brakes on that particular arms race.

At that time there was certainly a lot of interest in cruise missiles as nuclear weapon delivery systems. However, the technology did not mature to allow the same level of precision and reliability as ICBMs until about the early 1980s. One might imagine the incentive to develop fully intercontinental versions of advanced cruise missiles but there are several fundamental difficulties there. One is that you can't really shrink the size of a vehicle capable of delivering a missile across intercontinental distances much from the size of an ICBM. If you look even at airplanes with intercontinental ranges they are quite large. Worse yet, cruise missiles are much slower than ICBMs so they would travel for hours upon hours before hitting their targets (the Snark, for example, would take up to 9.7 hours to travel its maximum range). Even if they are radar evading due to flying at low altitude such long travel times are undesirable from the perspective of survivability. And undesirable from the perspective of determining mission success as well. If you need to order followup strikes, or followups to the followups, then you have an extremely drawn out process.

There is one way to shrink the size of an intercontinental range cruise missile though: using nuclear power. This gave rise to Project Pluto, the Supersonic Low Altitude Missile (SLAM) concept. Pluto/SLAM would have traveled at Mach 3 and used a nuclear powered RAMJET to fly an unlimited range flight to its target where it would either fly directly over the target and detonate an onboard warhead or it would drop bombs along the way (the design was still somewhat conceptual). The program was cancelled before it got very far partly because it was just hugely expensive, partly because ICBM technology (along with SLBMs and bombers) matured sufficiently during the 1960s to make it seem less necessary, and partly because it was just flat out insane (one of the problems is that it would inevitably leave a fallout exhaust from its reactor across whatever territory it traveled, such as the US, Canada, etc.)

So, overall the focus was elsewhere, the technological limitations were hard to overcome, and, as you point out, ICBM technology matured rapidly and its inherent advantages in terms of survivability, fast response times, constant readiness, etc. became abundantly evident during the mid 1960s, leading to ICBMs (and SLBMs) becoming the bedrock of nuclear launch capability. During the early scramble for nuclear delivery systems the ICBMs won the race, as the cruise missiles of the day just didn't match up. The Snark, for example, was never truly reliably enough to even deliver a multi-megaton nuke with enough accuracy to reliably destroy a target. Reliability issues in the form of "x% of the time it doesn't work" might be tolerable, reliability issues of the form "100% of the time it is incapable of fulfilling the primary mission" are not. As weapons development went through different phases during the Cold War there were opportunities for cruise missile technology to fill in gaps but the fundamental weaknesses were never truly overcome and the established systems had too much institutional/budget inertia to make room for a new nuclear delivery system megaproject with an unknown chance of success. Perhaps if the Cold War had gone on longer we might eventually have seen some sort of Have Blue / Snark / Tomahawk next generation intercontinental stealth cruise missile, but as history played out such long range cruise missiles just never seemed to find a niche.