Modified from an earlier answer

The Japanese non-invasion of the Soviet Union has its origin in both the labyrinthine decision-making process in the Japanese state and the immediate contingent situation facing the Japanese in late 1941, namely the American economic embargo on strategic war materials.

The grand strategic vision of both the IJA and IJN reached a major fork in the aftermath of the Russo-Japanese War. Without a common enemy (Russia) or a shared conceptualization of strategic space (Korea and Liaodong), the two services saw the future sources of Japanese power and security in different terms. The IJA favored a continental approach with a Japanese-led regeneration of the Chinese and the creation of various imperial satellite states within the East Asian mainland. The navy favored what became to be known as the "Southern Drive" for the natural resources and the seizure of the resource-rich lands of Southeast Asia. Both of these grand strategies had currency in the services prior to the Russo-Japanese War, but the interwar period saw this strategic thinking come into full flower. The IJN began to elaborate its southern strategy by designating the US as its main hypothetical enemy, while the IJA in conjunction with elements of the colonial apparatus in Liaodong created a puppet state Manchuria in 1931.

With regards to the Soviet Union, the IJA saw it as the inevitable great power threat to its ambitions in China. As such, the IJA devoted large resources to create a large field army in Manchuria, the Kwantung Army. Anticommunism became one of the chief hallmarks of the IJA officer corps and a number of officers longed for an invasion of the USSR. The IJN looked askance at these developments in China as a threat to its own domestic powerbase and as a diversion from any Southern drive. Inside Tokyo politics of the 1930s, the respective service ministries were able to take advantage of Japan's constitutional division of powers wherein the resignation of either the Navy or Army Minister could bring down a government. This created a political stalemate of sorts between these differing strategic visions.

The outbreak of war in China after the Marco Polo Bridge Incident did not really alter the strategic priorities for either service. The IJN used hostilities to seize Chinese territory like Hainan Island which could be used in a future Southern drive. The abandonment of the London Treaty system led to a new round of IJN construction, including warships designed to beat American quantitative superiority. The IJA for its part still kept up the establishment of the Kwantung Army and fought several sharp border clashes with Soviet forces. The IJA General Staff drew up two plans for a 1942 invasion of Siberia in 1939 at the behest of the Chief of the General Staff, Prince Ka'nin.

For their part, the Germans actually encouraged the Japanese to venture into the Pacific. Prior to Barbarossa, the German military planners in OKW had increasingly saw Japan as a vital counterweight against both the British and the US. A Japanese invasion into the Pacific would both tie up American naval power, thus preventing it from intervening in the Atlantic and by occupying SE Asia, Japan would be eliminating British access to the raw materials necessary for waging a modern war. Getting both Britain and America entangled in the Pacific would secure Germany's strategic rear while it took on the task of destroying the USSR. It was only well after Barbarossa had bogged down that German military attaches sought to encourage a Japanese invasion of Siberia, but by then the Japanese decision-making process had precluded such an action.

The stalemate about Japan's strategic direction broke in the summer of 1941. In an about face, the IJA gave a tacit support for the Southern Drive and the occupation of French-Indochina. The reasons for this volte-face were several. The German's swift occupation of Western Europe made these areas a highly tempting target. The bogged down nature of the war in China also made the IJA cast about for strategic alternatives that would deal the Nationalist regime a death blow. The bloody nose the Kwantung Army had received at Nomonhan in 1939 had underscored to the IJA that it needed a large-scale investment in heavy industry to modernize the IJA artillery and tank forces. These all factored into the IJA's strategic reorientation by mutually reinforcing each other. But what likely animated the IJA most in this strategic reorientation, and often reinforced these other factors, was economic pressures.

When the war in China commenced, the Japanese government created an entity known as the Planning Board to marshal Japanese natural resources and create an put Japan on an autarkic war footing.The problem with autarky, as their German counterparts discovered as well, was that is much easier said than achieved, especially for a nation with such limited resources as Japan. In 1939, the Planning Board took an unprecedented step and had actually cut resource quotas to both services. Although protests from both the IJA and IJN got these cuts restored, the lesson to both services was clear: Japan needed raw materials. The IJA's plans for the Siberian invasion called for a massive amount of resources and the Planning Board's 1939 decision forced the IJA to adapt the more modest of the two plans as the basis for its future planning. The advent of war in Europe added further pressures on the Japanese economy as it cut off Japan from European imports. This left Japan's only major source of foreign supply the United States. This drained Japan's foreign currency and gold reserves at an alarming rate. The Roosevelt administration's denial of export licenses and other embargoes in 1941 signaled to the Japanese military leadership that America was using an economic weapon to control the direction of Japan's security. The US made it relatively clear that the restrictions would be lifted when Japan modified its behavior. Both the IJN and IJA already were hostile to letting the other control the direction of Japanese security policy for the better part of twenty-five years, and now were certainly not about to let a foreign power do so!

But the experience with the Planning Board emphasized to both services the imperative need for Japan to acquire its own sources of strategic raw material. Not only was Japan facing a critical shortage of oil, but other materials like scrap metal needed to maintain a war machine. The shortages of scrap metal were so severe in 1941/42 that IJN ships were instructed not to waste too much ammunition in shore bombardment when conducting their invasions of Southeast Asia.

The common thread that ran through both the IJA and IJN's strategic planning was the need for raw materials. Although the IJA countenanced the IJN's southern drive, the General Staff still felt that the main strategic theater for Japan was the continent and that meant dealing with the Soviet Union. Plans for an Siberian invasion were shelved and the IJA detailed some of its best troops for the Southern Drive, but the IJA still maintained a large establishment in Manchuria. For its part, the IJN General Staff pushed for a non-aggression pact with the Soviet Union in Spring 1941 lest the Soviets take advantage of Japanese distraction. The IJA reluctantly agreed to the necessity of Soviet neutrality, yet it did not relish it. The IJA in turn adapted a wait and see attitude to the USSR after Barbarossa and the onset of the Pacific War.

It was both of those factors that destroyed any chance Japan would invade the USSR in foreseeable future. The lack of a Soviet collapse made it unlikely that either the IJN or the few remaining civilian leaders would accept breaking the treaty when engaged in operations in the Pacific. The scale of the Pacific War also put a damper on any thought of expanding the war. Japan had barely enough shipping to invade and hold Southeast Asia and the raw materials gained from the Southern Drive could not redress Japan's endemic shortfalls of strategic materials. The Kwantung Army and other formations in China found their best units and heavy equipment siphoned away into the Pacific between 1943 and 1944, so that the Kwantung Army was an operational husk of itself by 1945. Although the Japanese had more men under arms in China than at any time prior, these units were of marginal combat value and withered away when the Soviets invaded in August 1945.

Sources

Barnhart, Michael A. Japan Prepares for Total War: The Search for Economic Security, 1919-1941. Ithaca: Cornell University Press, 1987.

Evans, David C., and Mark R. Peattie. Kaigun: Strategy, Tactics, and Technology in the Imperial Japanese Navy, 1887-1941. Annapolis: US Naval Institute Press, 2012.

Miller, Edward S. Bankrupting the Enemy: The U.S. Financial Siege of Japan Before Pearl Harbor. Annapolis, Md: Naval Institute Press, 2007.