Land bridges and boats, generally.

During the ice ages the sea levels were lower, which exposed a lot more land above the water. This connected some parts of the world by land bridges and also caused some land to exist which is underwater today (such as Doggerland in the North Sea). That's how people got to the Americas, Hokkaido, the British Isles, Australia, etc. Sea travel was also a fairly important mechanism for colonization as well though. The Polynesians used boats to colonize numerous Pacific islands starting around 5 thousand years ago or so, for example. In fact, it may be somewhat surprising that Madagascar and New Zealand were populated in this way.

Additionally, there were waves of colonization for many locales, such as Japan, some occurring over land bridges, some by boats. For Japan in particular there appears to have been a wave of boat-based colonization around 1000 BC which seems to have been the major source of the modern Japanese population, with the pre-existing people who had migrated via land routes becoming a minority on the islands (being represented primarily in the Ainu today). There were also waves of immigration to the Americas over very long periods of time, some of which may have made use of boats, but we don't quite have enough evidence at the moment to make definitive statements.

It's important to recognize these patterns as the historical norm, because in large part they are. It's easy to look at the history of a place, ask the question "how did people get there" and then come up with an answer like "oh, they just walked there from Africa, case closed". The history of populations anywhere is rarely so simple, even in, say, Egypt which is incredibly close to the origin of modern homo sapiens. Most locales have experienced waves of migration and have convoluted histories. The British Isles, for example, have a very complex history of migration waves from neolithic tribes, from the celts, from romans, from anglo-saxons, from normans/norse, and more recently from all over the world. That situation is fairly typical around the world. In Italy you have waves of immigration during the neolithic, you have colonization from phoenicians, greeks, and others during the bronze age, you have migration during the roman period, you have migration during the post-roman period (gothic tribes, etc.) In Russia you have slavic tribes, you have norse migrations (kievan rus), you have byzantines, you have a period of mongol rule, etc. And these are places that we tend to think of as being very ethnically homogeneous as well (Japan, Britain, Italy, Russia), but that veneer overlays a lot of complex history.

By foot and early watercraft, as the periods in which the earliest homo sapiens sapiens migrations and subsequent inhabitation of all continents (except antartica) took place, transport animals like horses, dogs for sleds, donkeys, goats, camels, llamas, and all other transport animals were not yet domesticated.

Homo sapiens sapiens first migrated out of Africa via the Sinai corridor ~100,000 years ago, crossing into the Levant and into India, the Indies, and reaching Australia ~50,000 years ago. There were several migrations waves of humans from Africa, with another major out of Africa migration waves taking place around ~50,000 years ago.

Japan was settled by humans who first managed to cross from mainland East Asia either by a land route via a connecting prehistoric landmass between Korea and Hokkaido, or a martime route from China which may have involved island hopping. These first settlers were hunter-gatherers who first settled Japan ~35,000 ybp. These first settlers were later assimilated by a later wave of Korean migrants whose first presence in Japan dates back to ~900 BC. The Korean migrants seem to be responsible for the overwhelming share of modern Japanese ancestry, with >20% of Japanese ancestry being derived from the first settlers.

The first humans to crossing into the Americas were prehistoric Siberians who had mixed mostly East Eurasian (Eastern Asian, Australian, Pacific Islander-like) and significant West Eurasian (Middle Eastern, European-like) ancestry. The earliest migration wave of these Siberian migrants to the Americas began around ~20,000-16,000 years ago, crossing a prehistoric landmass that existed between Asia and Alaska called "Beringia", enabling them to migrate from eastern Siberia into Alaska. The descendants of these Siberian migrants into the Americas - the Paleo-Indians - began migrating southwards deeper into North America from Canada into the US, onwards into Central America and into South America up until the southernmost tip of the Americas (Tierra Del Fuego).