Exactly how you would expect, on ships or by walking. Looking at the Medieval Era and specifically Crusades, the European Armies had to march through Constantinople and Asia Minor in the First Crusade partially because there was no friendly port in the Middle East to land at. This would also make port cities crucial to bringing in armies and supplies throughout the time of the Crusader kingdoms.

Speaking of Napoleon, when he invaded Egypt he had a fleet of roughly 400 transports, which held 40,000 troops.