First of all, is this claim true? I've seen it on wikipedia, and quoted all over the web, but haven't found a reliable source for it.
Second, what happened to such people during Germany's occupation of Belarus, and what was their fate as that territory was reclaimed by the Soviets? Were slaves taken back to Germany? Were they only able-bodied men, or also women and children?
The Nazis planned on enslaving the Slavs in general, not just Belarusians. Like zhemao said, the plan was to starve out who you could and put the rest to work.
By the end of the war, there were 2.8 million Soviet ostarbeiter ("eastern workers") in Germany, 2.3 million of which were Ukrainian. (Orest Subtelny, “Ukraine: A History, 4th Edition,” University of Toronto Press, 2009).
“German planners envisioned the European lands of the Soviet Union as a colony, where German settlers would farm fields with the assistance of their Slavic slaves.” (Timothy Snyder, “Stalin and Europe: Imitation and Domination, 1928-1953,” Oxford University Press, 2014). According to Snyder, 380,000 Belarusians were “forced to serve as Ostarbeiter”.
Osterbeiter brought to Germany were both male and female. Children were included and worked up to 24 hours a week. (Harold James, “Family Capitalism: Wendels, Haniels, Falcks, and the Continental European Model,” Harvard University Press, 2006).