A novice reader/listener of documentaries, it seems that had Yamamoto instead focused all available ships/planes/equipment on Midway, the chances of victory would have been magnified. In part, the defeat was due to spreading our resources far too much, and leaving the carriers to fend for themselves. Why did the IJN have such complex strategies?
It was a combination of traditional institutional preferences for the IJN, and a fundamental misreading of the US by Yamamoto.
First lets look at how ideally the battle would have gone:
Suddenly and without warning IJN aircraft neuter Midway's air defenses and bomb ground defenses, this continues for 2 days with 2 or so waves per day.
At that point a heavy cruiser division would move in to provide gunfire support for a day followed by landings of troops to seize the atoll.
While this was happening the USN would be attempting to form a response and sortie from Pearl as they simply couldnt afford to lose Midway without a fight. When ready this force would likely include both American carriers and remaining battleships and possibly in two groups. It would be detected by picket lines of IJN I-Boats.
The Kido Butai and invasion force until now would have been the only force to engage US units, with the Main Body with the majority of the big guns was unknown the the Americans. In a lightening action the Kido Butai would destroy the American carriers and other ships as possible, and launch night attacks with destroyers and cruisers to fix their location. Then come the dawn the Main Body would arrive on the scene to finish off the remaining USN ships in close action.
This plan has its origins in several places. One is the proclivity of the IJN to plan for multiple moving parts, only semi mutually supporting forces, and multiple threat vectors. This was meant to not allow the enemy to concentrate against a single threat, remain dark as to the center of gravity, and ideally provide openings for fast decisive strikes. This mindset had worked well against the Russians, and been refined in the years since in action against the Chinese and in war games.
We then have the misunderstanding of the USN's mindset. It was perceived to be so broken as to be unwilling to give battle in anything but a quick raid save for key objectives. Hence Yamamoto's increasing fixation of sinking the remaining USN carriers after the Doolittle Raid meant he had to find a way to draw them out. Which meant he couldnt make the odds seem too bad so that they wouldnt be scared to come out, and do it in a place that couldnt be ignored.
This was simply not accurate. While the initial carrier raids in the Mandates, Wake, Rabaul, and Lae had all been quick in and out affairs they were not for an unwillingness to fight. Nimitz and King simply worked under the principle of calculated risk. As exemplified by Nimitz's orders to Fletcher and Nimitz at Midway, they were free to risk their forces so long as there was a good chance to inflict greater damage then they might lose.
And SIGINT allowing them to be positioned on an enemy's flank when unexpected was exactly the chance to inflict greater damage than might be risked.
Yamamoto thought that the American's would not come out unless his force appeared smaller than it was, while Nimitz was waiting for exactly that opportunity to strike a blow and relieve pressure in other areas, namely the South Pacific and sea lanes to Australia(where he had already risked 2 of his carriers and was prepared to gamble a third). So by placing his forces in non mutually supporting positions Yamamoto gave Nimitz exactly the chance that he was willing to take with his precious carriers.
Sources:
Shattered Sword by Parshall and Tully
Pacific Crucible Ian W. Toll
Black Shoe Carrier Admiral John Lundstrom