Given that Korea did have a partially recognized government in exile?
The Allies (US, UK, Republic of China - the USSR did NOT attend, due to the neutrality pact with Japan) discussed the disposition of Japanese colonies during the Cairo Conference in November 1943. They issued the Cairo Communique that stated that "in due course, Korea shall become free and independent." At the Yalta conference, Roosevelt got Stalin to agree to war with Japan. Stalin had conditions, however:
The Soviets entered the war two days after the bombing of Hiroshima and a day before the bombing of Nagasaki, and the Japanese surrendered a week later. The Soviets had built up an impressive force in the east and the Japanese were already battered and they knew the end was near, so the Soviets had very little trouble racing south across Manchuria. It was during this time that Dean Rusk and Charles Bonesteel (future Secretary of State and four star general, respectively) decided on the 38th parallel as a division in Korea, which the Soviets readily accepted.
The US had no way to get to Korea with the same speed. They had to come up from the islands in the Pacific, across thousands of miles of ocean with Japanese forces still in the area. It wasn't until September 9th that the US military accepted Japanese surrender in Seoul, well after the Soviets had taken over the north of Korea and 7 days after the signing of surrender on the USS Missouri in Tokyo.
As to why Korea remained partitioned: the Cold War.
The Allies had decided on a trusteeship for Korea. Korea would remain under UN control for a period of time (Roosevelt had imagined 40 years!) during which it would be rebuilt in every sense of the word: physically, economically, politically. After the trusteeship was over, it would be a free country. However, the Cold War got in the way. A nationwide election was boycotted by the North/Soviets (they knew they'd lose), both sides held separate elections, two governments were formed, and the rest is history.