As of 1939, a lot of people inside the borders of Poland were ethnically, linguistically or culturally Belorussian or Ukranian, particularly in the eastern parts. This isn't particularly surprising considering that most of this area was won from Sovet Russia and Soviet Ukraine in the Polish-Soviet war (in theory Soviet Russia and Ukraine were separate countries as of that conflict). The territory immediately below Lithuania was predominantly Polish, while the territory on the far eastern and southeastern reaches of Poland were predominantly Belorussian and Ukranian, respectively. This territory was home to large minorities of Poles, and depending on how you divide the areas up for study, you can find splotches of Polish majority inside the overall Ukranian and Belorussian areas.

Stalin considered Eastern Poland to be rightful territory of the Soviet Union. In 1939, in accordance with the secret protocols of the Molotov Ribbentrop pact, the Red Army advanced into Eastern Poland and took control. After annexing the Baltic States, Eastern Poland was administratively divided between Belarus, Ukraine, and Lithuania. In 1944-45, with Stalin reconquering all of Poland from the German Reich, the Eastern parts once again fell under their control. In Stalin's mind there was no possible question of returning this territory to Poland.

Although unwilling to return territory in the East, Stalin was willing to compensate Poland with territory in the West. This came in the form of German territory. Some of this territory was home to a large number of poles to begin with, but other parts were almost exclusively home to Germans, and so the new Soviet-Polish government began a program of deportation for Germans living in the new territory of Poland. The transfer of land from Germany to Poland killed three birds with one stone. It weakened Germany, it partially satiated revanchist Poles, and it made Poland reliant on the Soviet Union to guarantee it's new borders.

The second question - why did the Soviet Union take land from Germany - is a more straight-forward affair. The Soviet Union simply wanted a warm-water port in the Baltic. Konigsberg, now Kaliningrad, was free of ice all year round and therefore was (and is) a strategically significant port. Note that the Kaliningrad Oblast was populated by Russians because of its strategic significance, rather than being given to the Lithuanian SSR which neighbored it. The Germans who lived there were expelled much like the Germans in the new Polish frontiers, and Kaliningrad Oblast was populated by new Russian colonists. This is why there is a Russian enclave to the north of Poland and west of Lithuania today. The siezure of this territory was loosely justified as being part of the compensation given to the Soviet Union because of the terrible devestation wrought upon it by the Nazi invasion.

---

The main source for this is Six Months in 1945: From World War to Cold War by Michael Dobbs, which covers the post-war settlement and the conferences that went into creating it in a fair amount of detail.

There are two primary reasons that I believe are relevant here. The first and most obvious was strategic concerns. This is a more classic, realpolitik understanding of state behavior. Soviet efforts to control the Eastern Bloc, Central Asia, and East Asia can be understood as a means of providing a buffer against foreign influence and hampering potential military interventions as well as providing raw materials and manpower. The soviet domination of Eastern Europe is often focused on more than Soviet efforts elsewhere because of the extreme anxiety it produced in the West but the Soviets made significant, similar efforts elsewhere. This holds true well into the 1980s with the invasion of Afghanistan. Also remember, the Western powers sent military forces to aid White Russian forces during the Russian Civil War. The United States sent some 11,000 troops in total. The Empire of Japan sent 70,000 and Czechoslovakia sent 50,000. There were also troop contributions from France, Britain, Estonia, and Italy amongst others. Stalin wanted to ensure that such military intervention would be much more difficult in the future.

However from the angle of political legitimacy, Russian involvement in Easter Europe was largely justified to defend Marxist political thought and the logic of Marxist history. For Marxist-Leninists, the revolution was part of the inevitable march of history. The communist "revolutions" in Eastern Europe supported the narrative of the inevitable spread of the proletarian revolution. If the Soviet Union was the only major communist state, this would fundamentally challenge this historical narrative. Thus, the Soviet intervention in Eastern Europe was also a means of giving domestic legitimacy to the ruling communist party in the Marxist-Leninist sense. However, it was also a means of pointing to the power and prestige of the USSR eclipsing that of the Russian Empire. By controlling Eastern Europe and Central Asia (traditional spheres of Russian influence) then the USSR could argue that the revolution had empowered the Russian state to become more powerful and influential than Czarist Russia.