They could be brought. The thing is, a lot of scholarly literature uses the term "family" to refer to something more akin to a family firm that controls fairly extensive land and labor assets directly or indirectly-probably the two most famous Neo-Babylonian families are the Egibi and Murašu families, which were active for around 3-5 generations. The Murašu firm in general is a good case in point; although it is called a "family" it was active for several generations, spanned both the neo-Babylonian and Achaemenid periods(and contracted with the Achaemenids), and maintained offices both in Nippur(its original base) and the Persian city of Susa. We also know that it was less a family that "Went out and farmed" than a sort of corporation; it both made money by renting out land it leased in exchanged for providing a fixed stipend and rented land and cultivated it directly. A firm like this could afford to own slaves(although its labor force was not exclusively enslaved) and indeed we see documentation of slave ownership by this family and the Egibi family.