The toilet arrangements used on board Dutch East Indiamen – which typically would undertake the longest single voyages any ship would in the 17th century (as far as from Europe to Japan over a period of as much as two years, before the discovery of the faster southern route west across the Indian/Southern Oceans) may be of interest here, and I wrote about them in my book Batavia's Graveyard. Note that the weather conditions the ship was sailing in were of critical importance to the levels of comfort available in this regard:

There was no fresh water to spare for washing, and although one of the largest ships of her day, with a complement of nearly 350 people, the Batavia was equipped with no more than four latrines. One pair was located on either side of the Great Cabin, and reserved for the use of the people in the stern. The rest of the crew had to queue to use the remaining pair in the bow, which were nothing more than holes in the deck under the bowsprit. These heads were open to the elements and the gaze of all those waiting in line, and the urine and excrement deposited in them dropped straight into the water. The only additional amenity was a long, dung-smeared rope that snaked through the hole in the latrine. The frayed end of the rope dangled in the sea, and could be hauled up and used to wipe oneself clean.

This miserable existence was compounded in bad weather. All the gunports and the hatches had to be closed and battened down, and little fresh air penetrated below deck. The men stank of stale sweat and garlic (a popular cure-all at the time); everything was permanently damp, and it became too dangerous to venture to the latrines. Soldiers and sailors relieved themselves in corners or crouched over the ladders down to the hold, and if the weather was bad enough for the pumps to be called into action, the urine and faeces that had been deposited below made an unwelcome reappearance. Rather than discharging into the sea, the Batavia’s pumps simply brought up filth and water from the bilges, ‘fuming like hell and reeking like the devil’ as one contemporary put it, and sent it cascading down the gun deck to slosh around sleeping seamen until it found its own way out through open ports and sluices. When the weather finally improved, the men would scrub the decks with vinegar and burn frankincense and charcoal down below in an attempt to clear the air, but for much of the passage the lower decks of the Batavia smelled like a cess pit in a tannery. Those fortunate enough to travel in the stern were spared the worst of this, but every account of the journey east makes it plain that, during the first weeks at sea, even the most distinguished passengers endured discomforts they could scarcely have dreamed of back at home.

Have you ever heard someone call the bathroom "the head"? This is exactly where the term comes from!

During the age of sail sailors would most often relieve themselves on either side of the bowsprit, the long spar which sticks out above the figurehead, at the very front of the ship. As you can imagine, it can be a bit of a daunting prospect.

How bad could it be? You can see this in practice at the 18-20second mark in this clip from the movie Master and Commander. Yes, it is Hollywood. But it is an accurate depiction of the practice.

So you may ask, "why not go off the side railing or the rear of the ship?" Good question. When you are on a sailing ship, where is the wind most usually blowing from? Behind you. So that wind would be blowing up the stern, ensuring anything that gravity started carrying down was blown right back up again. Not a pleasant experience.

There also a few other options. It wasn't unheard of for a chamberpot to be used, but generally this would be more for non-sailor passengers - to include slaves who suffered the worst facilities of all.

A common mention in the records from these voyages mention the chamber pots provided to slaves to be too small for the number making use and for how often they were emptied. In addition, cramped conditions, shackles, and diseases such as dysentery often made a latrine pointless. The hold would simply need to be cleaned up after the fact. As you can imagine, this created a breeding ground for disease and was a major contributor to the high mortality rates of the slave trade.

As for earlier time periods, hopefully someone else can jump in.

Since there are a couple of good posts in here already, let me offer up some visuals (don't be afraid): Here are the heads on the Vasa, a Swedish ship that sank on its maiden voyage in 1628 and has since been salvaged and preserved. They're the square boxes on either side of the bowsprit. They likely would have had seats of some sort, but the seats haven't survived. This is a nice visitors' image of HMS Victory, showing the area under the heads. That would have been ... less pristine ... in the days of sail. And during Elizabethan times, at least, the English navy had a unique method of deciding who had to clean under the heads. (Citation: N.A.M. Rodger, The Safeguard of the Sea page 320).