There are two factors at play. As the size of the air group is increased, there was a real drop-off in the efficiency of spotting, launching, and recovering aircraft. The Midway class would prove this as it struggled with an air group as large as 130 planes.

More importantly, the Second London Naval Treaty was abrogated in September of 1939, which due to threat of war created a rush to build new ships. The USS Essex was approved for construction in 1940, with a planned completion of March 1944, although she would end up being launched at the end of 1942 due to accelerated priorities.

The most glaring problems with the Essex Class wasn't the size of the air group. They had some deck protection, but 1" less than that of the Midway Class. Torpedo protection was only slightly better than that of the Yorktown Class, although they did include alternating engine and boiler rooms.

The short answer is that the US Congress and Navy realized they needed several good aircraft carriers as soon as possible based on existing designs, rather than wait for supposedly perfect carriers several years after they were needed.

Source: Friedman, N., U.S. Aircraft Carriers: An Illustrated Design History.