Did the ideological divide cement itself in that short period of time even though western democracy and communism were new and foreign to most Koreans at the time?
As a side note, I grew up in Korea in the 70's. The Korean war and Japanese colonial rule were well covered in detail in school, but the period from 1945 to 1950 was sketchy at best. I know a bit of what went on on the south side of the border (US occupation, elections in 48, etc), but I am sure most of the world is still in the dark about what went on in the North. The DMZ at first was a nominal border at best, but as months went by, it became increasingly fortified which led my mother's side of the family to escape as they saw the writing on the wall.
Your comment, “the period from 1945 to 1950 was sketchy at best”, highlights a serious gap in the understanding of that period. But it’s incorrect to believe that this is by simple oversight, it was very much by design – particularly in korea. I would suggest that the fact that this is not truly in the past, i.e. the division of the peninsula is ongoing and there’s still an adversarial relationship between the two halves, and as a result it’s difficult to have something resembling an objective view of the period.
I’d disagree with the assumption in your question that it was the respective populaces that were politically divided and that this division was what led to the conflict. Rather I’d suggest that the respective governments were very much artificial and the result of the USSR and the US occupations in the context of the nascent cold war rivalry. To put it simply it was not the people of south korea and north korea that chose the respective governments, nor would they have even chosen Rhee SeungMan or Kim IlSung outside that context.
That obviously disagrees with the commonly understood narrative so in order to understand you’d need to go back to 1945, just after the end of WW2. The war ended but in korea the situation on the ground was no different and no US or USSR forces were on the ground to enforce that surrender. Despite that, once it was understood that japan had surrendered local populaces all over the country spontaneously organized town councils. These councils quickly and efficiently replaced the japanese administration, disbanded the hated police, and liberated political prisoners. These grassroot governments also cooperated at the macro scale, eventually leading to the Committee for the Preparation of Korean Independence (CPKI). The CPKI was in practice a national government and it was to it that the japanese colonial governor formally transferred control, eventually renaming itself the Korean People’s Republic (KPR). The use of “People’s” should not be understood to mean that it was a communist apparatus, it was a broad political coalition where all parties cooperated towards the liberation of the counry. The sole exception were those who would go on to form the Korean Democratic Party (KDP), at the time primarily comprising those who’d been embedded with the japanese and as a result had amassed great wealth as well as those who were irrevocably tainted by their deeds under the japanese, most prominently the police.
The replacement of the japanese administration with a korean one was complete well before the US forces arrived on the 8th of Sept ‘45. John Hodge, the head of US Government-In-Korea (USAMGIK), shortly declared the KPR an illegal and communist oragnisation. He reinstated the dismantled japanese colonial apparatus and generally speaking overtly favoured the collaborators, who now called themselves “anti-communists” and attached themselves to the US. It’s worth pointing out that in doing this he was explicitly going against State Department orders that USAMGIK was to remain neutral and not favour any particularly group or party. This included funding the formation of para-military forces under Rhee that were used to violently suppress “communists” (i.e. anybody who objected to the status quo) in the south together with the restored police force and the newly formed military. Jeju is a particularly salient example of this suppression. You can see echoes of this political violence in the language (e.g. calling the current government a “commie dictatorship”) and actions (e.g. violent attempts to break into the Blue House) of the “flag brigade”, most recently in their actions at the congress.
From what we understand, similar moves took place in the north. At first the soviet forces worked in cooperation with the KPR, most prominently with Cho ManShik. But eventually Cho would be replaced with the more cooperative Kim IlSung and political violence was used to cement that choice. We know from the flow of refugees from the north that forceful confiscation of land and wealth took place in the north. Many of the younger of these refugees went on to join or form para-military organizations in the south.
In regards to the war, nobody in korea accepted the notion of a division so it was no surprise that neither Kim IlSung or Rhee SeungMan did. Having cemented their local power it was natural that they’d see the unification of the peninsula, under their stewardship of course, as the next step. Both were extremely aggressive in their rhetoric and border clashes were common from both sides, so much so that the US denied the ROK army heavy weapons such as armour because they feared Rhee would start a war. Kim was of course not denied so it was not surprising that it was the north that started the war.
If neither the US or the USSR had been present in korea evidence suggests that the KPR would have continued on as the government of korea. Violence on the part of the collaborators was possible if not likely in order to maintain their place or hide their legacy but it’s unlikely that it would’ve succeeded. The division itself was the result of cold war politics so it’s difficult to see how it would have come about if neither country had forces on the ground. Eventually it might be possible for the remarkable level of cooperation within the KPR to break down but it’s unlikely that it would have led to outright violence rather than being resolved via political means. All in any conflict is likely to have been between collaborators vs everybody else rather than north vs south.
Having said that, once the war did happen it served to cement an enmity between the two populations, a conflict as vicious as the korean war tends to do that. But even then I'm not sure I'd see the enmity as rooted in politics per se. For instance, the south certainly wasn't democratic pre-1988 and the north is more accurately described as a monarchy than communist.
In regards to the lack of information surrounding the ‘45-50 period, this is due to the suppression of information in both ROK and the US. The primary account of the USAMGIK’s governance was written by the USAMGIK itself and alternative accounts were not given attention and within korea security laws suppressed scholarship of the period until the 80’s at the earliest. All in all it’s a period that sorely lacks attention.
I wrote more on the period in a few other posts, you can look up the sources I used for further reading/detail: Japanese collaborators and why they weren’t purged. Some repetition of what I also wrote here.