Few, if any, navies had good anti-aircraft weapons in the early 1930s. Japan's old naval AA gun was a dual-purpose 80mm gun (actually 3"/76.2mm), already in use during WWI. Later, they added the Vickers 40mm pom-pom, which was also used by the British. A typical early-1930s Japanese anti-air armament was carried by the heavy cruiser Atago: 4x1 12cm DP guns and 2x1 40mm pom-poms. While this looks quite inadequate by WWII standards, it was normal enough for the time. Comparable British and US ships were HMS Exeter (4x1 4" AA and 2x1 40mm pom-poms) and USS Northampton (4x1 5"/127mm AA). Of these, Atago had the biggest AAA armament.

In the mid-1930s, the Japanese adopted a new gun to replace the 40mm pom-pom. They wanted a gun with a higher rate of fire and higher muzzle velocity. They chose a 25mm gun. This achieved their goal; while it was fed by box magazines of low capacity and manual replacement of magazines basically halved the rate of fire, to only a little better than the 40mm pom-pom, it could be used in triple and double mounts giving a higher sustained rate of fire than the single 40mm mounts. It inherited problems from the 40mm: it used a manual mount and wasn't fast enough to effectively engage fast-moving aircraft unless they approached directly towards the ship, and the mechanical gunnery computer was poor and unsuited for targeting fast aircraft. It also suffered from muzzle flash making aiming more difficult. Basically, it was an improvement over the 40mm, but not a great improvement.

The British also felt the need for an improvement on the 40mm pom-pom. Their choice was the 40mm Bofors. The 1903s USN light AA gun was the M2 0.50 machine-gun, and they felt the need to improve. Their choice was a 1.1"/28mm gun, which was used in quad mounts to achieve the same rate of fire as the M2 0.5. By the time this gun was in use in significant numbers, the USN felt it was inadequate, and replaced it with the 40mm Bofors and/or the 20mm Oerlikon. The 40mm Bofors was a better gun than the Japanese 25mm. The Japanese 25mm, the US 1.1", and the 20mm Oerlikon were comparable, with the 20mm Oerlikon the best of the three.

During the war, the Japanese navy fitted large numbers of the 25mm gun to its ships, but didn't adopt a better gun to replace it. First, they thought that the 25mm was effective. The Japanese navy doesn't appear to have analysed the performance of its AAA weapons as thoroughly as other navies did. Even if they did, it would have been difficult to replace it. It would have required (a) designing and making a new gun, (b) making ammunition for it, and worst of all, (c) making new gun mounts. The biggest problem with the 25mm gun was the slow mount - manually-powered, it was too slow to track fast aircraft, and even powered versions were too slow. An all-new powered mount would have been necessary for a real improvement. Realistically, this was beyond the capability of Japanese industry, which struggled to supply enough ammunition for the navy's existing gun - the navy adopted AA fire doctrine restricted the amount of ammunition used as a result.

Instead, the Japanese opted for quantity. The above-mentioned Atago finished its career with 4x2 12.7mm/5" DP guns and 4x3, 8x2, and 32x1 25mm guns (for 60 25mm guns in total), and 4x1 13.2mm machine guns. The Japanese navy kept up quantitatively, but still suffered from inferior computers. In comparison, US heavy AAA was much more effective than Japanese AAA, due to the proximity fuse. Also, late in the war, USN fighter cover was much better. Thus, the USN depended less on its light AAA. Japanese ships, on the other handed, needed their light AAA, and found it was not effective enough.

/u/wotan_weevil has already done a fantastic job covering the story of the infamous Type 96 25 mm AA gun. Ultimately, of course, the issue lies in the fact that Japan's industrial base was woefully insufficient to support the kind of war that Japan embarked upon.

However, I wanted to focus on a few other aspects for why Japan is perceived as having such weak anti-aircraft armaments, as well as questioning the narrative somewhat.

Perhaps the most prominent reason is that the Japanese simply did not stress anti-aircraft fire as an air defense mechanism as much as the Allies do. It is critically important to remember that the goal of naval air defense is not to shoot down enemy aircraft. Rather, it is to prevent the enemy's munitions from striking your ship. If you can do that by striking an aircraft out of your enemy's inventory, then so much the better, but if you only succeed in throwing off the enemy's attack run or otherwise force them to take an unfavorable approach that results in a miss, then that counts as a success for the ship's air defense. With this in mind, the Japanese placed a far greater emphasis on maneuver as a key piece of their air defense system. For a Japanese carrier, the ship's helm was as important an anti-aircraft defense weapon as any anti-aircraft weapons the ship might carry. In comparison, the Americans chose to favor firepower in terms of their anti-aircraft defense. This meant that--in addition to increased spotting times due to a lack of radar--Japanese naval formations tended to be much looser than their American equivalent. While American formations stressed the need to avoid excessive maneuvering in order to maximise the effectiveness of their anti-aircraft guns, Japanese formations provided much more room for their carriers to maneuver. What this did also mean was that, at the moment of decision, often the carrier's anti-aircraft defense weapons were the only ones which could actually engage an incoming aircraft, as opposed to the denser American formations, where Japanese pilots would often be subject to fire from a number of escort ships even on their final attack runs.

Secondarily, and this is an issue that everyone struggled with, is anti-aircraft fire control. Fire control is an arcane science at the best of times, and naval anti-air fire control particularly so. Not only is the target, small, fast-moving, and moving in three dimensions, but the firing platform itself is moving in three dimensions too, as the ship rocks and rolls with the ocean even before adding in any maneuvers the ship might be attempting to pull off. Ultimately, the American solution to the naval fire control question involved both radar guided fire control and the famous proximity fuse. At the outbreak of the war, the best Japanese optical fire control anti-aircraft fire control systems were on-par with or even superior to their Allied counterparts. However, the Japanese were never able to incorporate radar directed fire control into their anti-aircraft weapons, and even during the sortie of Yamato in Operation Ten-Go, Japanese anti-air fire control remained optical in nature, without radar guidance. The failure to develop and incorporate radar, despite their excellent optical fire control systems was yet another example of Japan's economic base being insufficient to truly support a military capable of fighting the war Japan set out on.

Finally, while the Type 96 25 mm AA gun was infamously poor, not all of Japan's anti-aircraft weapons were bad. In fact, the Japanese fielded what was probably one of the best heavy naval anti-aircraft guns of the war in the Type 98 10 cm gun. This weapon was mounted on the dedicated anti-aircraft Akizuki class destroyers, as well as making up the anti-aircraft batteries on the carrier Taihō and the submarine command cruiser Ōyodo. The Type 98 had a higher muzzle velocity and higher rate of fire than the famous US Mk 12 5 inch gun. The Type 98 was a very effective anti-aircraft weapon, but--like with many next generation Japanese weapons--it simply was not available or deployed in sufficient numbers to have a meaningful impact on the deciding issues of the war, nor did its effectiveness do anything to resolve the far deeper underlying issues with the Japanese war effort in general or the Imperial Japanese Navy in particular.