The concept of worlds beyond earth with other life is rather ancient, even more ancient if you count heavenly beings as otherworldly.
Some of the oldest literature of traveling to other worlds and finding life that I know of is in the Book of Enoch, which describes stars as fiery mountains burning for their sins. However it's unclear if it's referencing stars themselves as being beings, or worlds full of beings, burning for their sins. Either way, it was one of the earliest movements from stars being holes in a sky dome, to being physical places you can visit. It's rather grim:
And came to an empty place
And I saw there neither a heaven above nor an earth below, but a chaotic and terrible place.
And there I saw seven stars of heaven bound together in it, like great mountains, and burning with fire.
At that moment said I, “For which sin are they bound, and for what reason were they cast in here.”
Then one of the kodesh malakim, Uri'el, who was with me, guiding me, spoke to me and said to me, “Enoch, for what reason are you asking and for what reason do you question and exhibit eagerness?”
“These are among the stars of heavens which have transgressed the commandments of YHWH and are bound in this place until the completion of tens of thousands of years, according to the number of their sins.”
I then proceeded from that area to another place which is even more terrible and saw a terrible thing: a great fire that was burning and flaming; the place had a cleavage that extended to the last sea, pouring out great pillars of fire; neither its extent nor its magnitude could see nor was able to estimate.
At that moment, what a terrible opening is this place and a pain to look at!
Then Ura'el, one of the kodesh malakim who was with me, responded and said to me, “Enoch, why are you afraid like this?” I answered and said,
“I am frightened because of this terrible place and the spectacle of this painful thing.” And he said unto me, “This place is the prison house of the angels; they are detained here forever.” -Enoch, 21
This is from sometime around 300BC-100AD.
A far more detailed and explicit mention of aliens and worlds they inhabit comes from the 2nd century work of dramatic fiction known as "A True Story" by Lucian of Samosata. Lucian details a war between the people of the moon and the people of the sun fighting for colonizing rights over Venus. However, it should be noted that Lucian himself says he wrote the book as a criticism of mythology. He was writing as an atheist using science fiction as a display of the ridiculous. He himself laughed at the idea as ridiculous and unreasonable, writing:
My subject, then, is things I have neither seen nor experienced nor heard tell of from anybody else: things, what is more, that do not in fact exist and could not ever exist at all. So my readers must not believe a word I say