The overwhelming majority of territory lost by Germany in comparison to the Weimar Republic borders was to Poland (yet small segments were annexed into the USSR as well). Of course, Austria and what was once the Sudetenland were also liberated/reannexed. This was, in part, a sort of political revenge against nazi expansionism throughout the world war, but the reality of the territory loss was somewhat harsher. In short, Austria maintained its German speaking population, the territories reannexed into Czechoslovakia and annexed for the first time into Poland and the Soviet Union were... not as lucky.

Germany's modern borders, much like the borders of East Germany, border Poland along two rivers, the Oder and the Neisse. This was an ethnically arbitrary line drawn as the new German-Polish border after the German surrender. The two pronged outcrop and exclave you can see on the maps of Weimar Germany, were surrendered to Poland despite their populations being majority German. I will, despite being a collective of regions with many names, refer to this region as East Prussia from now on.

In 1939, when the Soviets and Germans partitioned Poland, a rather massive part of what had up until then been Polish territory became part of multiple soviet constituent republics, Ukraine and Belarus benefiting the most, though eventually with the integration of the Lithuanian S.S.R., they also benefited by receiving Vilnius, which had been a part of Poland before the Soviet invasion. This left the Soviets, already abundant in ethnic conflict and with a relatively harsh policy in maintaining ethnically homogeneous constituent republics (except of course for Russians, who were and are notable minorities in many post-soviet states), with an undesirable polish population.

The annexation of East Prussia into postwar Poland was, in effect, making room for the eventual population exchange. Of course, there were other major processes behind the decision to annex such vast swathes of territory, such as compensation towards the polish communist government, humiliation of the aggressive expansionism of the nazi regime, and, again, to make room for the Poles coming in from the regions annexed by the USSR in 1939.

I have thus far neglected to talk about the Sudetenland, which was the Czechoslovak border region with Germany, annexed in 1938 by the Germans and reannexed postwar by the Czechoslovaks. This region, before the war, housed 3 million ethnic Germans, who were, much like the Germans in East Prussia, an absolute majority in their region, soon to be treated to a harsh expulsion. The motivation for these deportations however was less pragmatic than making room for incoming refugees. The Czechoslovak prewar president, Edvard Benes, who had spent the war in refuge in England, representing the Czechoslovak government-to-be, had effectively negotiated not only to retain the pre-1938 borders of Czechoslovakia, but also to guarantee a lack of future ethnic conflict in the region. Due to significant nazi elements in Sudeten German society prior to the Munich agreement (which had seen the annexation of the sudeten region), they were effectively seen as having collaborated with the nazis, and sentenced to collective expulsions.

These losses of territory were the most major. Overall, the allies oversaw the expulsion and relocation of over 10 million ethnic Germans. This was accompanied by expulsions from nearly all of Eastern Europe where German exclaves and ethnic islands had existed: Hungary, Romania, and in a spectacularly brutal fashion Yugoslavia all saw the removal, be that through deportation or pogroms, of the German population. These latter ones were, by and large, without much pragmatic consideration, simply revenge for nazi occupation.

Ask more if you'd like to know more. Main source was Orderly and Humane: the expulsion of the Germans after the second world war by R. M. Douglas.