This is a really fun question! From my understanding, which may be incorrect, or lacking nuance provided by others, it largely stems from post-WWII immigration and relocation practices.

The final months of WWII saw massive amounts of displaced persons, either those fleeing the battles at the front, workers from across the German Reich who had been moved to Germany for labour, or German settlers who fled their settlements. Finally, at the close of the war, there were massive amounts of prisoners, either POWs, or those from Nazi camps.

The response of the Western Allies, across their Zone of Occupation, was largely to return these people to their country, regardless of the individuals needs or wants. Workers from the East, either from the Soviet Union, or Poland were returned to the Soviet Union or Poland. People from the Balkans were returned to the Balkans. Germans from Sudetenland were moved back into Germany proper. The Western Allies simply put, returned people back to their ethno-geographic homeland.

As for Eastern and Southern Europe, this process wasn't put in place to the same extent. Also the descent into war during the breakup of Yugoslavia, displaced further refugees into the more modern era.

I am aware that this answer is simplistic at best, and doesnt examine other factors. My source for the answer was the excellent book by Tony Judt, 'Postwar: A History of Europe Since 1945', which I believe is on the askhistorians reading list. If you would like a more in depth view of the question I highly recommend obtaining a copy, if not just for how amazing this book is! Chapter III and IV are especially relevant in regards to ethnic restructuring and placement of Europe.