I Recently watched “War in the Pacific - Eagle against the sun” and it said the IJN’s view on submarine were to serve adjunct to the battle and attack fleets. How did they rationalize this all while seeing their Allie in Nazi Germany decimate merchant fleets?
The Imperial Japanese Navy (IJN)'s submarine doctrine was an adjunct to the grand plan for a Decisive Fleet Battle that would decide the outcome of the war in a single titanic fleet engagement. As you correctly note, the role the IJN envisioned for its submarine forces was one focused much more heavily on attacking enemy warships, rather than merchant ships and supply lines. There are a few key reasons for this, one of which is doctrinal. Leadership within the IJN was well aware that Japan stood at a significant disadvantage when compared to the United States. Beyond the 5:5:3 ratio imposed by the Washington Naval Treaty, the Japanese were well aware that the US had a massive economic advantage, and so the IJN would always be fighting at a numerical disadvantage. Thus, the IJN's primary focus was how they would--as a smaller weaker force--overcome the larger, theoretically more powerful American fleet. A key part of the plan developed was for an attritional campaign as the USN advanced westward across the Pacific, and submarines would play a key role in this, along with land based aircraft which could be shuttled from island airfield to island airfield as the battlefield shifted further west.
This doctrine played a major role in shaping IJN submarine operation in two major ways. First, it defined the primary target of the IJN's submarine force as warships. The IJNs submarines were meant to target warships, to help whittle down the strength of the American fleet, so that when the Decisive Battle came, the battlelines of the USN and IJN could face each other at odds where Japanese victory could be possible. The second, is a bit more deeply buried within the doctrinal approach, and indeed lies with one of its fundamental assumptions, namely, that victory in the decisive battle would effectively end the war, and a negotiated settlement would follow. This necessarily meant that the war would be short, and primarily decided by a single major campaign and action, where the primary burden would be on warships. Logistics would simply not be as essential a factor, as the campaign and war themselves would be short. There simply wouldn't be time for a sustained campaign against merchant shipping to have any effect, and so the IJNs submarine forces focused themselves into preparing for attacks against warships. Sinking a merchant vessel wouldn't have a major impact on the decisive battle, but even damaging a battleship could be the difference between victory and defeat.
Another major factor, heavily related to the above, is institutional. Quite simply, due to the doctrinal emphasis on offensive action, talented and driven officers and men wanted to be placed in combat units capable of delivering those blows. They did not want to end up in something so devoid of a potential to strike a blow against the enemy as logistics, and--indeed--the IJN's logistical capabilities were terrible throughout the war, only declining more as US submarines became more effective. Operations planning groups similarly were thus filled with officers who had experience as surface and aerial warfare commanders, but did not have much in the way for logistical considerations. Any skilled office would want to earn glory and prestige in a combatant command, not languish in something as tedious as logistics. Indeed, throughout the war, the Imperial Japanese Navy seemed to operate under the attitude that whatever the Navy did, the logistics people would simply find a way to support it. The IJN wanted to viewed warfare almost entirely through the lens of combat and what was directly required for bringing about tactical victories. Victory was determined by warships sunk, not logistical capabilities, and the leadership of the IJN in turned pushed for operation that would result in sunken warships, not merchantmen. This was clear even from the very beginning of the war, with the attack on Pearl Harbor. The logistical capabilities of the base were simply not a topic of concern for the Japanese planners, and that was as true for the submarine forces as it was for the aviators.
As an adjunct to this, it is not as if the Japanese were unaware of the potential for submarine campaigns against commerce. During the First World War, a unit of Japanese destroyers had gone to the Mediterranean to aid the Allies in escorting convoys there. This Japanese force proved very effective at its mission, to the extent that the Royal Navy requested that more Japanese crewed ships be deployed for convoy escort duty. While this assignment did provide the IJN with ample experience in anti-submarine warfare, as well as and understanding of how important their mission was. However, at the end of World War I, there was little appetite within the IJN for topics as non-essential as "convoy escort" or "commerce raiding," and the experiences of these crews was mostly put aside. The IJN was far more concerned with preparing its ships and sailors for the great battle with the American navy, rather than anything else.
To summarise, the IJN simply did not conceive of its submarine forces as a weapon to be used against merchant shipping. The submarine was a tool by which they, as a weaker force, could help even the odds against a numerically superior enemy, and the design and training of submarines and their crew placed emphasis on this. The Japanese viewed their submarines as a tool for sinking warships in advance of the Decisive Battle, not as a tool for a slow strangulation of commerce. If asked why they were not targeting merchant ships, as Japanese submarine commander would likely respond by questioning why he would waste one of his torpedoes on a merchantman when he could be saving it for a battleship or aircraft carrier. It was simply not part of the doctrine or institutional culture of the IJN to focus attention of the non-glamorous side of war.
I hope this has helped to answer your question, and please feel free to ask any follow ups.
Sources