Interesting question, and quite a big one! For starters, the Japanese did not actually intend to conquer all of China during the Second Sino-Japanese War.
But, to truly understand the scope of the Second Sino-Japanese War and why the Japanese could not conquer all of China, you must go back to the start of the conflict: 7th July, 1937.
To give a proper answer is going to take way more than 10,000 words, so I'll split it into three parts: the background of the Second Sino-Japanese War, the opening stages of the war, and the later stages.
Part I: On the 7th of July, 1937, the Marco Polo Bridge Incident served to incite a full-scale Japanese invasion of China. Japanese forces quickly overwhelmed the token Chinese forces in the area and seized control of parts of northern China that had hitherto lain outside of their control - including the city of Beijing. The Japanese central government was, at this point, content with the gains made, and did not wish to escalate the situation in China to all-out war.
The primary ruling party in China at the time was the Kuomintang (KMT), led by Jiang Jie Shi, or Chiang Kai-shek. Prior to the outbreak of war, Chiang had been slowly building up the military and industry necessary to fight Japan on equal footing. Arguably, Chiang was purposely avoiding full-scale war and confining the Sino-Japanese conflict to "incidents" so he could build up China. By 1937, he was arguably still not fully prepared, but he needed to make a tough decision. If Chiang continued to make concessions to the Japanese, it would allow him to keep building up resources, but would severely erode the already-weakened public support of the KMT, that had thus far allowed Japan to chip away at China's territory piece by piece during the aforementioned "incidents", including the Marco Polo Bridge Incident. Yet, if Chiang chose to fight a full-scale war, he risked losing all the considerable assets that he had spent years building up.
However, after the fall of Beijing following the Marco Polo Bridge Incident, Chiang decided that China had finally reached its boiling point; enough was enough, and China would tolerate no further Japanese aggression. To not make a stand against Japan at this point was to be utterly unpatriotic. It was time for war. The battle to decide the fate of China, Chiang decided, would be fought in Shanghai.
Why Chiang chose Shanghai is a question of much debate, since he risked losing the nascent industrial base he had spent years building up in both the city itself and the surrounding areas. Perhaps it was because he risked losing the industry there that he had to fight in Shanghai? For one thing, Chiang's newly-organized, German-trained military divisions were barely ready to meet the enemy. For another, the Japanese could easily reinforce Shanghai in the event of an attack via both sea and land routes. One reason could be that Chiang's intentions were not only the defense of Shanghai, but the larger Jiangsu Province, where both Shanghai as well as the then-capital of Nanjing were located. Jiangsu Province was one of the few places in China where the central KMT government had unopposed political authority; other parts of China were arguably under the control of regional warlords. Chiang therefore possibly intended to use Shanghai as a key defensive choke point against the Japanese. Another reason could be that the river-rich Jiangsu Province denied the the Japanese armor of their shock advantage. He could also have been intending to use Shanghai to garner international attention, as there was a large expat population living in Shanghai even back then.
Shanghai's situation at the time was also quite complex. After a treaty following the January 28 Incident in 1932, the Chinese arguably controlled the city, but were not allowed to station troops in both the city and the surrounding areas; only a small police force was allowed to be stationed in the city by the Chinese. The Japanese on the other hand were allowed to maintain a small garrison of 6,000+ marines in the city. Subsequently, the police force in Shanghai was trained in unorthodox military tactics unusual for a police force, and they played a big part in the battle to come.
It is also important to understand the Imperial Japan's structure was somewhat disunited. The Army, the Navy, and the central government, were all operating on their own wavelengths to some degree. For example, the Army was interested in consolidating its holdings in northern China and did not wish for an escalation of war in Shanghai, but the Navy stubbornly insisted on keeping its holdings in Shanghai.
On August 13, 1937, Chiang ordered Chinese forces to start entering the city in force. The first skirmish had the Chinese police force in Shanghai exchanging small arms fire with the Japanese marines, before 70,000 of Chiang's elite German-trained troops arrived to reinforce them. The Army, hitherto unwilling to be committed to the fight, was now forced to participate in the face of the Chinese attack. The Battle of Shanghai had begun.
Part II: The aim of the attacking Chinese forces was to deny the small Japanese marine garrison in Shanghai the use of the wharves which the Japanese could use to reinforce. However, the vastly-outnumbered Japanese marines entrenched themselves, and managed to hold their position with the use of naval bombardments and air support.
Lacking heavy weapons, the Chinese infantry were unable to break through the Japanese positions, many of which had bunkers that they could not destroy directly. Chinese tanks brought in to help destroy entrenched Japanese forces were vulnerable to Japanese anti-tank weapons as they were lacked infantry support, for the Chinese did not have sufficient training in coordinating infantry and armor. The few infantry divisions that kept up with the tanks were wiped out in close-quarters combat where the Japanese employed flamethrowers and machine gun nests. The Chinese assault ground to a halt, and vicious street-to-street fighting in enclosed urban areas led to heavy casualties mostly suffered by none other than Chiang's vaunted German-trained divisions. The product of years spent trying to modernize China's miltiary were dying by the hundreds.
The Chinese fortunes took a turn for the worse in August 23, when Japanese reinforcements arrived in force. The Chinese had failed to destroy the Japanese defensive positions in time, and now, superior Japanese troops were arriving to claim Shanghai. At the height of the Battle of Shanghai, 700,000 Chinese troops faced off against 300,000 Japanese. Instead of a lightning offensive to seize Shanghai as had been initially expected, the Chinese were now forced to fight a defensive war of attrition. However, Chinese forces put on the defensive were every bit as stubborn as the Japanese marines who had held out against them, and the Japanese counterattack was a slow and arduous process. The potential fallout from losing Shanghai was too high, and the Chinese refused to give ground. Despite being armed with superior weaponry, and backed by naval, aerial, and artillery support, it took the Japanese months of intense fighting before they managed to conquer Shanghai. On October 26, 1937, the Chinese started retreating from the city of Shanghai itself.
Chiang had hoped the international Nine-Power Treaty held during Battle of Shanghai would call for sanctions against Japan, but it was for naught. By November 8, Chinese forces who had been fighting in the surrounding areas of Shanghai were ordered to retreat back to the final defensive lines before the capital of Nanjing (the Wufu and Xicheng Lines). Exhausted Chinese forces who had been fighting for months on end emerged from the bloodbath at Shanghai only to find the civilian officials posted at the Wufu Line had fled and taken the keys with them, rendering the defenses unusable. The Wufu Line fell on November 19, and the Chinese retreated to the Xicheng Line, which fell on November 26. The fortifications around the capital which the Chinese government had spent precious millions on had fallen after only two weeks.
Fighting continued unabated even after the fall of the Xicheng Line. The Japanese, hoping to seize the initiative, wished to use the impetus of their victory at Shanghai to seize Nanjing as well. On November 20 the Chinese made the decision to defend Nanjing. Conflicting statements from Chinese high command, including from Chiang himself, over whether Nanjing was to be defended to the death or merely used as a delaying action made troops confused at the lower levels. Chiang probably knew what dire straits the Chinese forces were in, as he officially moved the capital from Nanjing to Wuhan, a city which far up the Yellow River.
The road to Nanjing was marked with small delaying battles while Chinese forces worked feverishly to construct hasty defenses both inside and outside the city. The fight for the city of Nanjing would soon begin earnest when the Japanese reached the final line of defense outside of the city, the Fukuo Line, on December 9. The Japanese requested that the Chinese forces surrendered in the face of the overwhelming strength of the Japanese forces, but received no response. The next day, on December 10, the Japanese decided to attack.
As the Japanese attack was about to commence, Tang Sheng Zhi, the Chinese commander in charge of the defense of Nanjing, proclaimed "Each unit shall firmly defend its post with the resolve to either live or die with it. You're not allowed to retreat on your own, causing defense to collapse!"
The mood of the Chinese garrison was mixed. The Chinese defenders consisted of about 3/4ths raw, hastily-recruited troops of poor quality, and 1/4th the elite German-trained divisions who had fought a retreating battle all the way from Shanghai, including the crème de la crème of the Chinese forces, the Training Brigade (so named because they were responsible for passing on their western training to other Chinese divisions).
For the elites of the Chinese military, their attitude was one of hopeless, unyielding defiance. Frank Tillman Durdin, who was reporting on site during the Battle of Nanjing, reportedly witnessed a small group of Chinese soldiers in their defensive position assemble in a solemn semicircle, where they promised each other that they would all die where they stood. The Battle of Nanjing had begun.
For the untested Chinese troops of poorer quality, it was probably much more terrifying. Tang Sheng Zhi was absolutely clear on his orders that no man was to retreat one step back without orders. Similar to the USSR's situation in the Battle of Stalingrad, it is said that over 1,000 Chinese soldiers were shot by their own side for attempting to retreat or desert. Chinese defensive positions, such as pillboxes, were chained shut from the inside to prevent the defenders from fleeing.
As the battle begun and the Japanese attacked in force, Chinese defender retaliated with great ferocity, either out of personal valor or because they had been forced into such a position with no other way out. After weeks of vicious fighting similar to Shanghai, on December 11, Chiang ordered Tang to abandon the defense. There was no time to coordinate the retreat in time, however, and when Tang escaped on December, some units never received his message to retreat and hold on to their positions to the bitter end, while others disintegrated and were in open flight.
By this time, the situation was very clear: Chiang's gamble had failed completely, and he had lost all that he had feared he would lose. Both Shanghai and Nanjing had fallen, his elite German-trained divisions including his very best Training Brigade had been all but obliterated, and his armor and air force had been wiped out at Shanghai. The Chinese were well aware that the Japanese had the overwhelming advantage in sheer military prowess, so it was in this dark time for China that Chiang decided on the strategy of "trading space for time".
Part III: After the fall of Nanjing, the Japanese Army was confident that China would fall in three months. They were counting on a fast conclusion to the war. Meanwhile, the Chinese were already preparing for a war of attrition. The Japanese prediction was largely based off estimates of the KMT's existing military forces prior to the war, and how long it would take to crush them. These predictions did not particularly take into account the forces and manpower that lay outside of direct KMT control, such as the Chinese Communists, or the rebuilding of the forces that were shattered in the opening stages of the war.
The next year, in 1938, the Japanese central government was hoping to preserve their strength and limit the conflict to the areas around northern China, Shanghai, and Nanjing. By this time, however, the central government had effectively lost control of the Japanese Army. Emboldened by their victories in Shanghai and Nanjing, Japanese forces pursued the retreating Chinese. Prior to this, the Japanese Army had kept close to the northeastern Chinese coastline, as well as to the shores of the Yellow River; this meant that the Japanese Navy could maintain supply lines to the Army and the Chinese forces were unable to do much more than lay water mines. This also meant the Japanese Army were much better-supplied than the Chinese forces, despite Chinese forces fighting on home terrain. This was because much of China was not yet modernized, lacking highways and railroads, and large numbers of troops and materiel could not be moved across China without difficulty.
Ironically, the Japanese Army, in their rush to wipe out the Chinese forces that were in retreat from the northern theater after Shanghai and Nanjing, ran into the same difficulty that the Chinese Army faced. Leaving their maritime supply lines behind, the pursuing Japanese Army successfully encircled but failed to fully contain Chinese troops at the Battle of Xuzhou, allowing many Chinese troops to escape into the countryside where they later fought an effective guerilla campaign against the Japanese. At this point, the Japanese Army was now too far away from the maritime supply lines they had established, and despite supplies being air-dropped to them the Japanese soon found themselves under-supplied and stranded deep in enemy territory. In the ensuing Battle of Taierzhuang on April 1938, the Japanese forces were defeated despite the Chinese lacking the conventional weapons required to defeat Japanese armor and heavy weapons; the Chinese successes against Japanese armor is attributed to the establishment of Gan Si Dui (敢死队), or literally "Dare to Die Squads", who wielded swords and wore suicide vests made of out grenades that they detonated underneath enemy tanks. The myth of Japanese military invincibility had been broken, providing a great increase in morale for the embattled Chinese forces.
Shocked by the defeat of the Japanese Army at Taierzhuang, the Japanese alternated their strategy to proceed with caution. Instead of chasing down the Chinese forces deep inland where it was hard for the Japanese forces to resupply and where the Chinese had the numerical advantage, the Japanese Army resolved to stick to their maritime supply lines and launch offensives from there. All of the Japanese Army's existing forces in China proceeded along the Yellow River to capture Wuhan, which was the second-largest city in China at the time, and being the new provisional capital of China after the fall of Nanjing, was of great economic, military, and political importance. Unable to defeat a well-supplied Japanese army, hard-pressed Chinese defenders ultimately failed to defend Wuhan, which was captured by the Japanese on October 27, 1938. Despite the fall of Wuhan, and Japanese demands for Chiang to come to the negotiating table, Chiang refused to negotiate unless the Japanese agreed to his outrageous demands to withdraw to pre-1937 borders. When the Japanese refused, Chiang simply relocated his forces to Chongqing, a city deep within inland China, which became the new provisional capital.
By now, the Japanese knew that to launch a major offensive inland would leave them bereft of their supply lines and probably lead to a repeat of a disaster like the Battle of Taierzhuang. Unwilling to risk such an offensive, and frustrated with the Chinese simply moving their base-of-operations/capital elsewhere whenever it was captured, the Japanese resorted to massive aerial raids that specifically targeted civilian areas in major Chinese settlements. Millions were killed, injured, and otherwise left homeless. By doing this, the Japanese hoped they could pressure Chiang into a surrender, but their actions simply built up the resentment of the Chinese populace against them.
This resentment manifested in a series of successful Chinese attacks where the Japanese were defeated at Suizhou, Changsha, and Guangxi. The Japanese forces had expected China to be in no state to continue fighting after Wuhan, and were badly surprised by the Chinese offensives. Then, from 1939 to 1940, the Chinese sustained a prolonged counter-offensive campaign against the Japanese that ultimately failed, but caused the Japanese to suffer 60,000 to 70,000 casualties - a steep loss in manpower that Japan, an occupying power, simply could not afford.
The turning point of the war would come in 1941, when the Japanese launched their attack on Pearl Harbor, causing the USA to join the Allies and declare war on Japan and the rest of the Axis powers. Within days, China also joined the ranks of the Allies, and Allied support of the Chinese war effort begun in earnest. After this, Japan would gain no more ground in China, and steadily lost territory in China until their surrender to the Allies in 1945.
Wilson, Dick. When Tigers Fight: The Story of the Sino-Japanese War, 1937-1945. Penguin Group USA, 1982.
Dorn, Frank. The Sino-Japanese War, 1937-41: From Marco Polo Bridge to Pearl Harbor. 1974.
Henriot, Christian. "Beyond Glory: Civilians, Combatants, and Society During the Battle of Shanghai." War & Society 31:2 (2012): 106-135.
F. Tillman Durdin, "All Captives Slain," The New York Times, December 18, 1937, 1, 10.
Fenby, Jonathan. Generalissimo: Chiang Kai-shek and the China he lost. Simon and Schuster, 2003.
Hsu, Immanuel Chung-yueh. "Rise of modern China." (1975).