The Japanese had two doctrines the first was mainly supported by the Japanese army named Hokushin-ron (Northern Expansion Doctrine) which mainly focused on expanding north and at one point capturing Siberia for its rich resources. When Japan invaded Mongolia, they unofficially fought a war with the Red Army and the war didn't go too well for them which led to Japan and the USSR signing a non-aggression agreement. After the defeat in Mongolia the Japanese Navy, which mainly supported Nanshin-ron (Southern Expansion Doctrine) where Japan would focus on island hopping for resources to fuel its isolated empire.

So in short, the reason the Japanese didn't open up another front was because of their previous experiences with the Red Army, and the Imperial Japanese Military believed they had a lot better options. Not to mention, the Imperial Japanese Military also wanted to completely take care of all Pro Communist Partisans in China before they would even try to take on the Soviet Union.

just expanding on the answer from /u/Han_Zulu somewhat...

The key to this question is understanding that the alliance between Germany and Japan during the Second World War was tenuous at best. Nothing even like the cooperation between the UK and the USSR, it was merely an agreement between two countries who had similar expansionist aims, though in different areas. Throughout the war very little resources or even information was exchanged. The Japanese didn't even tell Hitler about the Pearl Harbor attack in advance.

Conversely the relationship between the Allies was much closer. The UK and the USA setup a joint command structure - the Combined Chiefs of Staff (CCS). And strategic planning meetings between the leaders of the UK, USA and the USSR were held regularly.

As has been mentioned, the Japanese experience with fighting Soviet troops at Khalkhin Gol in 1939 had reduced the desire for additional expansion in this area somewhat. Furthermore invading the Soviet Union would have required a significant allocation of army forces. Forces which were already having issues controlling the situation in China and were busy occupying Manchuria. Meanwhile the navy, which had been built up at great expense, would be unable to assist in this option.

Paradoxically the navy found itself arguing for the Southern option even though several high-level naval officers, including Yamamoto, were of the opinion that Japan could not win a protracted war against the United States. Yet to admit that would be to concede that any money spent on the navy was wasted and that the army should receive the bulk of the military budget.

In the end, the Strike South option won. The Germans weren't even consulted.

source : Ian W. Toll, Pacific Crucible

Just want to reemphasise one point that the other posters mentioned, which was that the IJA was busy in China. The IJA had half of it's total forces stationed in China and were having great difficulties making progress.

Southern Expansion was targeted at the resources and wealth of South East Asia, and in particular the Dutch colonial possessions, rich in oil. From 1940, the United States had cut off scrap metal shipments and the use of the Panama Canal from Japanese merchants, and in 1941 this was extended to an embargo on oil trade with Japan. This former was painful for Japanese industry, the latter was potentially life threatening for the Japanese Empire.

Thus, the idea of attacking the USSR in 1941 instead of the United States and Western Allies is problematic from a strategic standpoint, because it would mean dramatically extending its military commitments in mainland Asia. ~80% of Japan's oil came from the US, so the prospect of waging a large, oil hungry war against the Soviets without any way to replace American imports would be a recipe for disaster.