In antiquity people rarely travelled straight through the sea or ocean as this was extremely dangerous. The main method of navigation was sailing along the shore line with ocasional island hopping to prevent disorientation. This was because people in antiquity did not have acces to compases, coordinate systems or reliable navigation maps to be able to safely sail across large bodies of water so they relied on masses of land as landmarks for orientation.

Navigation across the open ocean/sea was usually on short distances such as from Carthage to Sicily for instance and in this case, navigators relied on watter curents and winds to carry them across the ocean or sea.

When out in the open sea celestial navigation was used and in Europe it is first recorded to have been used by the minoans.

As for exploration, explorers followed the shore line, watter curents or wind curents and see where it would bear them, this is how the arabs and indians reached each other across the open sea by following the monsoon winds across the Indian Ocean.

EDIT because i forgot: Of course it was not fault proof, expeditions often ran out of suplies and where forced to turn back home, for example the greeks made a series of expeditions in east africa and the phoenicians explored east and west africa both in an attempt to circumnavigate it but decided to turn around because it was too dangerous, see the expeditions of Hanno and Eudoxus of Cyzicus.