This answer is limited to the ancient Greek and Roman world.

tldr: Speculation about distant lands quickly followed the notion of a large spherical earth.

In the Archaic period the first theories about the size and shape of the earth started to appear. The known or inhabitable world was called the Oikoumene and was thought to be surrounded by a great ocean. The earliest source of this notion is the Illiad (Book 18, 476-609). Here the shield of Achilles is described which was heavily decorated and depicted an allegorical map of the world. The great river Oceanus forms the outer rim of the shield. The idea of a great landmass surrounded by an ocean probably remained dominant through the earliest developments of geographical thought.

In the 6th century BC Pythagoras (or Parmenides) is thought to have been to first to claim the earth was a sphere. Parmenides further developed the theory of different climate zones. Around the equator there was a 'burned zone', to the north and south of that the tropical zone, further towards the poles a temperate climate zone and, on the north and south pole an arctic zone. This view implies the possibility of other inhabitable lands far to the south of the known world. But the distances were to great and the equatorial zone was deemed impassable.

About a century later, in Plato's Phaedo, Socrates says: "[...] the earth is round and in the middle of the heavens [...] I believe that the earth is very large and that we who dwell between the pillars of Hercules and the river Phasis live in a small part of it about the sea, like ants or frogs about a pond, and that many other people live in many other such regions." (Phaedo, 109a-109c). The Greeks and almost all of the people they were in contact with were situated around the Mediterranean sea. The growing understanding that the earth was exceedingly large compared to the inhabited world probably influenced the idea of other, faraway lands.

During the Hellenistic period geographical knowledge expanded both to the west and to the east. In the late 4th century BC Pytheas of Massalia made a voyage beyond the pillars of Hercules and explored the Atlantic coasts of Europe. He visited Britain, the Baltic, Maybe even Norway and Iceland, commonly referred to as the mythical northern land of Thule. Carthaginian knowledge of the Atlantic Ocean (both off the coasts of Europe and Africa) was much more expansive than what the Greeks knew. Tales of distant islands (probably the Madeira islands or the Azores) reached the Greeks through the exploits of Carthaginian or Phoenician traders and explorers. In the east the conquest of Alexander the Great, and after that the expansion of the Seleucid Kingdom, contributed heavily to the fuzzy Greek knowledge about the mysterious Persian lands and the lands even farther to the east. Contacts with people from Indian states and indirect knowledge of eastern (Chinese or Southeast Asian) civilisations indicated the earth was very large.

In the 3rd century BC, Eratosthenes wrote extensively on geographical topics. He thought it was very likely there were more inhabited lands in the northern temperate zone. During the transition between the Roman republic and the Roman empire, the Greek historian and geographer Strabo wrote the Geographica, which summarised and expanded all previous geographical knowledge. Other Greeks and Romans (most notably Ptolemaeus in the 2nd century AD) further cemented the knowledge that was gained during the past centuries.

In conclusion, there was definitely speculation about faraway lands as early as the 6th or 5th century BC. But it remained speculation. Philosophers and geographers had a theoretical framework from which they constructed a (sometimes surprisingly accurate) view of the entire earth. They knew it was certainly possible that there were other inhabitable parts beyond their reach, since their calculations showed the earth to be very large compared to the known world of Europa, Asia and Africa. But as far as we know, these faraway lands stayed beyond their reach. Mentions of distant lands were mostly allegorical or mythical (for instance Atlantis and Thule).

Main source:

Roller, D. W. (2015). Ancient Geography. The Discovery of the World in Classical Greece and Rome.

​