We do know that the Hammurabi Code, our oldest known written legal code, had rules for slaves. So 4000 (ish) years ago slavery was an accepted established part of that society. That takes it back to the beginning of written history.

Certainly there is evidence for slavery practiced in almost every part of the world in some form or another in pre-historical cultures but for answers on prehistoric times you are probably better off heading over to /r/askanthropology.