So I've read that Leo Szilard and Isidor Rabi convinced Enrico Fermi not to publish his discovery that purified graphite worked as a cheap neutron moderator. Because of this, the Nazis didn't know and had to use deuterium, which was much harder to acquire and stopped by the Norwegian heavy water sabotage.
Is this true?
A full nuclear program would have taken up a sizable percentage of any country's industrial capacity, and none of the Axis powers had industrial capacity to spare on such a project. The Japanese Navy's Committee on Research in the Application of Nuclear Physics, chaired by Dr. Yoshio Nishina, concluded in 1943 that "it would probably be difficult even for the United States to realize the application of atomic power during the war".
If you look at the breakdown of the Manhattan Project budget only about $795 billion million out of $21,600 billion million went into R&D (using constant 1996 dollars). The majority of the money goes into the diffusion plants.
What do diffusion plants do? They take a chunk of mixed Uranium isotopes and separate them into U-238 (which isn't useful for weapons, at least not until more advanced ones were made) and U-235 (which is what they wanted). Since U-238 is far more common in nature, it takes a lot of work to purify Uranium ore to the level of "highly enriched weapons-grade Uranium" (90% U-238235).
Now, modern purification methods use centrifuges, which is about 10 times better than the old diffusion methods. Since diffusion methods were what was best-developed at the time, and the Manhattan project had less time than it had money, they went with diffusion. Sure, centrifuges might have worked out eventually, but by then the war would have taken its course.
Short of a breakthrough in using centrifuges, it's unlikely either Germany or Japan would have been able to produce a weapon by mid-1945. Even with centrifuges, they would have needed to divert precious resources from other production efforts, thus hastening their own demise.
The Germans never really pursued their fission program with any earnestness. They had a small reactor research program that was focused on developing reactors for propulsion than as fissile material breeders. Even then, they saw this as a long-term program, not something to produce short-term war results.
If they had really wanted good moderators they could have built heavy-water plants in-country (it is not that hard — the Manhattan Project built three four heavy water production plants that they didn't even use, and it only took up about 1% of the total project funds — $27 million in 1945 USD). If they had really pursued graphite at all they would have seen that it was just a purification problem. The common denominator here is not a technical issue but an organizational one: they were not adequately committing to the work to the degree you'd need to get adequate short-term results, and thus not really confronting any of the problems in an all-out manner to the degree that was necessary to have a success on par with the Manhattan Project.