All Germans from beyond the Oder-Neisse line ended up in West Germany, because as the front was getting closer, they fled, because they were afraid of the Soviet troops. They had no reason to stay in the Soviet occupation zone afterwards, so they settled in the West. The situation was different in Czechoslovakia, where they were expelled systematically, but again only a negligible number of Sudeten Germans went to East Germany, because they knew what was the best for them.

After that the ones who made the mistake and stayed in East Germany had a plenty of time to change their mind - the Berlin wall was built only in 1961, so they could comfortably escape through Berlin, and many of them did.