Were they doing slave labor? Or were they doing more suitable tasks like being advisors or scribes?
As a corollary to OP's question: What was the life of an average Jew in Babylon like? How were the classes divided? Were Jews well integrated into Babylonian society?