Why did the Axis powers (particularly Imperial Japan) fall so far behind the Allies on information security/code breaking?

by Hard_on_Collider

There's two parts to this question:

  1. Why did neither the Japanese nor Germans successfully regain infosec despite their codes being cracked so early in the war? Both Enigma and Japanese naval codes had been cracked by 1942, allowing the Allies vital insight into Axis war plans. The Battle of Midway seems the most egregious example, with the Americans wiping out the Kido Butai with only 3 carriers thanks to the element of surprise. Did it occur to no one at high command that their codes had been breached?

  2. Were there equivalent Axis successes in breaking Allied codes? I'd imagine that Japan and Germany would've had some teams working to break the Allied codes. How were there no similar breaches made by the Axis?

I'm asking mostly for Japan, because most previous questions focus on Nazi Germany.

NotSoButFarOtherwise

Germany's main deficiency in cryptanalysis during the war was that every government office (telephone/telegraph, mail, police, all three branches of the military, etc) that captured messages was also responsible for the cryptanalysis thereof - there was no equivalent to Bletchley Park or Arlington Hall. This led to a lot of duplicated effort, but also meant that the most talented cryptanalysts couldn't always be put on the most difficult problems. It's no surprise, then, that if the Germans couldn't always effectively share cryptanalysis resources within their government, they wouldn't be set up to share with their allies, either.

As far as I know Japanese cryptanalysis didn't play much part in the course of the war. Part of Japan's weakness came from the fact that they were very late to develop modern cryptosystems; through the 1930s, Japan relied on Polish cryptographers for expertise and training, most likely due to shared interest in breaking Soviet codes. Equally important, however, may have been the fact that the United States communication in the Pacific before 1942 used weak, probably known-to-be-brroken encryption schemes and not infrequently none at all. To some extent that may have been deliberate, to discourage Japanese intelligence from investing more heavily into cryptanalysis capabilities or to enable ruses (such as the "AF no water" trick employed at Midway), but it simply may also have been due to woeful underinvestment by the Americans in the interwar period.

Before the outbreak of the war, U.S. Gray, Brown, and Black codes were compromised, either by cryptanalysis or by intercepting the code tables (or a combination thereof). Among other things, this enabled Germany to monitor U.S. diplomatic communication between Washington and Europe, North Africa and the Middle East. However, the ECM Mark II (SIGABA) machine introduced after the U.S. entered the war stymied all cryptanalytic efforts (that we know of) during the war, as neither it nor the British Typex were broken. This is probably important: the Enigma machine had been in use by Germany since the 1920s, and while iterative improvements had been made in the interim, there was already a wealth of theoretical cryptanalysis on Engima by 1939 that gave the Allies a head-start in code-breaking; similar applies to the Japanese "Red" code and its successors "Blue" and "Purple".