Teddy Roosevelt is credited as making the first foreign visit as a president-elect. And Taft is credited for the first visit by a sitting president, in 1909. Even after that, travel remained rare. Few presidents made visits abroad during their time in office. And those visits tended to be to places like Mexico, Canada, and Cuba (or the Bahamas for a vacation). Wilson of course made his famous visit to Europe as part of the 1918 peace conference, which was a huge deal at the time. A sitting president had never been to Europe before. And it wasn't until the late 1950s or early 1960s that it became somewhat common.

Prior to the 20th century, travel simply took too long. 90 days was common for New York to London in 1800. And even 70 years later, 30 days was a pretty standard transit time. It is only with the adoption of self propelled ships that travel time changed enough to make it practical. Likewise the invention of the radio and transoceanic telegraphic lines allowed a president to remain at least in some level of contact with his White House staff back home. So it isn't surprising that sitting presidents mostly stayed in Washington.

In the age of rail they had Presidential trains for long distance travel around the USA. There were special carriages that would make up the train, complete with bedroom and offices just like Air Force One now. Foreign travel was of course less common as it took longer but still happened, the President taking advantage of luxury passenger liners, especially across the Atlantic. During times of war military ships were often used, though by the time of WW2 this was considered too dangerous at times (due to the U-Boat threat) or too slow and an aircraft option was used.

Wasn't the whole point of having a Secretary of State and Ambassadors to foreign countries that leaders could stay at home and lead while maintaining diplomacy?