It was definitely more than the rivalry we would see at, say, a US Army vs. Navy football game. If I recall correctly, it had its roots in the Meiji restoration, where the victorious Imperial faction, led by Choshu domain and Satsuma domain, set up the government and military departments. Choshu was more focused on land, while Satsuma was focusing more at sea. There had been a historical rivalry between the two ever since the battle of Sekigahara, where the Choshu domain (led by the Mori clan) and Satsuma domain (under the Shimazu clan) ended up blaming each other for allowing Tokugawa to win, Elements of this rivalry ingrained themselves thus between the army and the navy.

If one were just to consider the expansionist factions of the IJA and the IJN, these two groups of militants had very different goals as to how to expand the Japanese Empire. The IJA wanted to lead the way by invading China and/or Siberia, a primarily land operation. The IJN was more interested in invading south, and seizing the Dutch East Indies, a primarily naval operation. Needless to say, both army and navy were interested in essentially being more powerful than the other.

Unlike most of the other powers of WWII, Japan did not have an independent air force. Both the army and navy controlled their own warplanes and pilots and jealously guarded them from the other. This obviously led to extremely poor coordination where many Japanese forces were deprived of air cover unnecessarily, and joint Army-Navy operations were almost always extremely disorganized. Consider Guadalcanal, where the Army decided to work with the Navy to convey troops piecemeal against the heavily armed and supported US 1st Marine Division. It was a pretty poor showing by the Emperor's finest, to say the least. This is not to say that all operations ended poorly-for example, see Singapore-but there were many operations that definitely could have been conducted with more finesse.

The situation was so bad that the Army and Navy developed naval and ground components respectively to exert influence. The Japanese "marine" component in Shanghai was actually just naval personnel trained for ground operations. Similarly, the Army commissioned the Akitsu Maru, a light aircraft carrier (although she was more of an aircraft ferry) as well as several other ships. This division and misallocation of war industry definitely hurt Japanese war production.

Towards the end, as the Japanese suffered defeat after defeat, both sides ended up blaming each other for the conduct of the war. They both tried to one-up each other-this led to much waste of life, like Operation Ten-Go, where the navy, wanting not to be outdone by the Army's contribution to the defense of Okinawa, threw its last operational assets in a failed suicide attack just to save some face with the government and the Emperor. After that, the IJN was finished, with most of its senior officers killed or wounded, its forces non-existent, and its ability to contribute to Japan's defense virtually nil. The army took full control of the war from that point (not that it wasn't already in power) as the main remaining Japanese forces were all in the Army.