Admiral Yamamoto knew that a confrontation with the U.S would mean certain defeat for the Japan. Originally the plan was never to fight the U.S in a prolong battle, but to cripple the American fleet at its Anchorage (Pearl Harbor) and mop up the remaining remnants in a decisive fleet action. Their plan, I would argue was greatly influence by Mahan's view of Naval power, where a great power could bring their fleet to bare in a decisive action and inflict such crushing defeat that the the enemy would capitulate. Japan's plan didn't take into account other innovations that were occurring around the same time. Their carrier based attack (Which in itself was inspired by the British) meant that the days of two opposing fleets slugging it out were over, now the center piece of any battle fleet would be the Carrier, which Japan was not able to target (The American Carriers were out to sea at the time).

With Japan being unable to deal a crushing defeat, Japan essentially engaged in a prolonged loosing battle, with an opponent that could outproduce it in every item/weapon.

Part of Japan's success in the opening days of the war was also due to an over extended British empire. Even before the war, the Royal Navy knew that it could not defend all parts of the empire against a well equipped enemy. When war broke out, England faced the German fleet (Uboats included) in the Atlantic, the Italian navy in the Mediterranean, and the full might of the Japanese imperial Navy in the Pacific. Simply put, there weren't enough ship to station everywhere they where needed. When you look at their campaign in the eastern empire, you see that they first targeted the two capital ships that England had sent to the "Great Fortress of Singapore", and they were sunk with ease. With the ships gone, Singapore and to a great extent, the rest of the eastern empire was open for the taken.

The United States in the other hand was a two ocean navy, and with war looming, it began to move its capital ships to pearl harbor. In many ways the attack on pearl harbor signaled their own defeat, as they conclusively proved that a carrier force could inflict sufficient damage on an opposing fleet.

As has already been said, the Japanese high command didn't think they could outright beat the US. That wasn't their goal.

Basically, the IJN subscribed to Alfred Thayer Mahan's school of military theory which, among other things, advocated a single decisive battle designed to crush an enemy fleet.

If you look at Japan's post-Meiji Restoration history actually bore out that Mahanian decisive battle idea. It bore fruit in the First Sino-Japanese War when the IJN smashed the equally modern and numerically greater Beiyang Fleet, and in the Russo-Japanese War when the Russian Baltic Fleet was destroyed at Tsushima. In both cases Japan was able to count on the discontent generated among their enemies domestic factions by these crushing defeats to sap the will to continue the war.

This was the sort of effect Japan was counting on- they knew they couldn't win a long drawn out war so they tried for a short shocking campaign and failed.

One perhaps can't fault Japanese institutional belief in the idea of a decisive battle being an effective tactic- a force multiplier if you will- against a more powerful enemy- the trouble is that unlike Qing China and Tsarist Russia, the US didn't have the sort of political and institutional fragility that would lead it to cut a deal with Japan in order to divert resources to quashing domestic threats to the government, and combined this with the sheer industrial might to make losses ultimately irrelevant. It's not possible to win a meaningfully decisive battle against an enemy who can replace any tonnage you can meaningfully sink in under a year when you can't replace your own losses.