I am currently reading "A Fiery Piece in a Cold War", which discusses the development of the U.S. ICBM program. In it, the author briefly discusses how Bernard Schriever, one of the key U.S.A.F. officials involved in the ICBM program, went to see John von Neumann at the Institute for Advanced Study at Princeton (in 1953) to discuss whether it was possible to fit a 0.5 - 1 MT yield warhead into a missile (which would have required a weight < 1 ton). The book notes that both von Neumann and Teller confirmed to Schriever that by 1960, such a warhead would be available. We now know that warheads meeting those specifications were indeed created.
What is known about how nuclear physicists managed to improve the yield to weight ratios of thermonuclear warheads to go from the CASTLE BRAVO SHRIMP device (which weighed > 10 tons) to a warhead weighing less than a ton and yet retaining megaton yields? Obviously, I understand quite a bit is not going to be known. But I was wondering what we do know.
And in case it's not obvious, I would love a u/restricteddata answer :)
Obviously there is a lot we do not know, because H-bomb design is itself still pretty classified (despite some declassifications and some leaks), and miniaturized H-bomb design is, you know, pretty classified, since that is what a state like North Korea or Iran would find the most useful.
But my understanding is that the improvements you are talking about, between 1954 and the early 1960s, that allow you to have relatively compact H-bombs that can fit on ICBMs (usually as single warheads) are not because of any really radical innovation. Rather, they came from just taking a very deliberate approach to optimizing the existing designs so as to squeeze as much bang out of your device as possible. So, for example, a really obvious approach is to go over every non-nuclear component of your bomb and try to figure out how you can reduce its weight, and to go over every nuclear component (like tampers and pushers) and try to figure out how to make it contribute more to the explosion. One really obvious (if very expensive) way to do this is to make the pusher of your secondary out of highly-enriched uranium, for example. And for the really small ones in this time period, they did things like make the reentry vehicle casing part of the warhead, to avoid duplication of walls and thus weight. And there are the tried-and-true approaches like those already taken in the late 1940s and early 1950s in doing things like improving primary design and efficiency with new high explosives, new lens and firing set setups, new approaches to core shells and air gaps and so on — things that squeeze just a little more efficiency out, but no "big" ideas that change everything.
All of this kind of design work is pretty boring — just shaving off percentages, working the efficiency downward bit by bit. This is more or less what Livermore in particular got very good at, just essentially auditing every part of the warheads to see how you could squeeze out a little more from it. If you do that, you can get pretty far along the line in terms of improving your yield-to-weight ratios.
The only really "radical" innovations appear to have come in the early 1960s, with the DOMINIC test series, and those involved pretty far-out, different approaches like RIPPLE that are still pretty classified, but we know that the people at the time thought they were something pretty brand-new. These were really tricked-out designs that had things like super-light spherical secondaries with shaped radiation wave pulses for the compression — the kind of stuff that emerged, interestingly enough, out of work in other areas of nuclear physics (like inertial confinement fusion). These were sort of the last major innovations in weapons design, apparently; work from the 1970s onward was "polishing a turd," as weapons designers sometimes called it, back to the efficiency-chasing and safety-improving mindset.
Anyway, that's the story as I know it, but the reality could always turn out to be more complicated. The main book on these things is Chuck Hansen's Swords of Armageddon, which tries to chronicle all of the innovations for (literally) every warhead in the US stockpile. One thing I would point out is that of course, one man's "efficiency improvement" might be another man's "amazing innovation," depending on how one wants to frame it. My sense is that the two main periods of "big innovations" are the late 1940s/early 1950s, and the early 1960s — these are when most of the new tricks and clever new designs are developed, with the periods in between being characterized as being somewhat more workman-like. But again, all of this is through a lens of classification, so there's no doubt more to learn.