For fear of violating the 20-year rule, while I going to use existing statistics, I am interested in long-term factors which could contribute to the strength of the Japanese communist movement. I am sure that these factors, whatever they are, were in place prior to 1997.
The Japanese Communist Party boasts more than 300,000 members and appears to be surging. Though nowhere near dominant, it is more than a third of the size of the ruling Liberal Democratic Party. After the Democratic Party (not to be confused with the LD), the Japanese communists are the second-largest opposition party. Why is this the case?
I understand that Japan has a very old and rich communist tradition, but so do Germany and France. In Germany, there is no substantial communist party. The closest would be the hard-left Linke, a medium sized party which is declining in membership. France, the nation which arguably birthed the socialist movement, has approximately 70,000 dues paying communists. The only non-ruling communist parties with more numbers than Japan are those in Russia and India, countries with entirely different economic circumstances and entirely different histories with communist ideology.
Did Japan never have a red scare? Japan is an industrialized country and also a capitalist country in Asia. These two factors are generally not conducive to the strength of left-wing movements, as they have usually declined in the former and been suppressed in the latter. So how do the Japanese communists survive and prosper, despite the odds?
Did Japan never have a red scare?
Holy shit buddy, boy did they ever!
To understand contemporary Japan's communist party, you have to remember that contemporary Japanese democracy was fundamentally shaped by reconstruction, and therefore by the United States.
It's the end of WWII. The U.S.-occupied transitional government jails all the Meiji capitalists and tries some of them for war-crimes. This "purge," was designed (as expressed in the Potsdam Declaration, the 1945 terms of surrender) to "eliminate for all time the authority and influence of those who have deceived and misled the people of Japan into embarking on world conquest, for we insist that a new order of peace, security and justice will be impossible until irresponsible militarism is driven from the world." The Purge was exceptionally wide-ranging, and removed right-wing leaders in education and political parties, but also the heads of industry which had been complicit with the state. Like most occupying forces trying to set up a transitional government, the U.S. looked initially towards installing the opposition of the ancien regime to replace the conquered government. In this case, the U.S. looked towards to Japanese Socialists who had been jailed and suppressed by the militaristic, right-wing Meiji government. In the first election of the new democracy in 1947, the Social Democratic Party won, which led in part to the creation of a Japanese constitution with left-leaning guarantees such as an article in the constitution guaranteeing robust rights for workers to organize (Article 28).
At the same time that Japan's left wing was surging back to life (helped along by the decapitation of the right-wing political parties, the popular dissatisfaction with militarism, and the U.S.'s backing of leftist parties), the U.S. was abruptly changing its position with regard to the Japanese left. The U.S. realized that in order to wage war in Korea, it would need to restart Japanese industry to supply vehicles and munitions. This necessitated returning control of industry to the very same capitalists and militarists which had been purged during the first stage of occupation. Secondly, the U.S. began to view the left as being sympathetic to its new greatest threat, the U.S.S.R. These two factors led to MacArthur's policy shift against the left, and the beginning of Japan's "Red Scare," also known as the "Red Purge."
Through 1949 and 1950, more than 20,000 left wing journalists, teachers, ministers and politicians were expelled from their employment and prohibited from political participation. These were primarily members of the Communist party. Some of those purge victims are still suing the government for redress today.
It's too late for me to summarize the next several decades of Japanese history, but broadly, it is a history of the left being violently suppressed by the right-leaning Liberal Democratic Party and the more explicitly militarist/nationalist fringe. One stunning and visceral example of this was the assassination of Inejiro Asanuma, who was killed by a nationalist with a sword while giving a speech live on national TV. ((CW: NSFW - that link is a video of the murder). There were major clashes throughout the 1960's between socialists and the State, most notably those protesting against Japanese support for the U.S. military, and the continued presence of the U.S. military in Japan.
The mystery is not "why is there now a strong communist party," but rather "what kept one from developing sooner?"
For more see: Allied Occupation of Japan by Eiji Takemae