They did. Both the Greeks and Romans had large cargo ships that were powered purely by sail with no oars. According to the OCD these larger ships could often have multiple masts and even triangular topsails. These are the ships that we find in ancient shipwrecks with their precious cargoes, as no preserved warship from antiquity has ever been discovered. It's important to note that warships also were powered purely by sail when not in bad weather or in battle. The Cambridge Classical Companion notes that "power sailing," or using both oars and sails simultaneously, was not used in the ancient world (except among a few types of very very small ships, like Liburnians), though it must have been known (we have a lot of mosaics and so forth depicting ships with both oars and sails out, so at least for artistic purposes it was understood to be possible). Warships in antiquity, particularly smaller ones like triremes, were not particularly seaworthy, being designed for swift travel and for battle. They had to be beached at night and could not adequately cross large distances without making landfalls, thus the practice of island hopping or remaining near the coast (a practice which the large cargo ships did not have to follow). Sails and oars were known to easily unbalance a ship, and Antony was considered notable for having his sailors bring their masts with them at Actium, stowing them in the hold rather than leaving them on land.