If no clean water was available, they used sand or dust, and probably still do if water is scarce. It even has a name, Tayammum.

The main population centers of the Muslim world were in Egypt and Iraq, and remember that both places are famous for their river systems and they had advanced irrigation systems that had been in place for millennia. So for most Muslims in the majority of the Middle Ages, water was not a rarity.

Muslims living in the deserts of the Arabian Peninsula or in the Maghreb may have had problems with water, but as /u/Implausibilibuddy points out, Muslims could use other material for water instead.

Ablution, or washing up with a ton of water is relatively modern, thanks to seemingly unlimited access to water via a tap. Ablution is sufficient even at the least amount of water, provided that it covers the areas it should cover. If you consider that, it really isn't that hard to do it with the equivalent of a small water bottle.