Well for the most part it relied on intermediate languages. Travel was slow and often when you stopped you could find someone who spoke a language you could understand, plus the language of the people just a little bit further down the river. For example when the Lewis and Clark needed an interpreter who could speak Shoshone, they hired a Frenchman whose wife could speak it and another language. She would relay it to the Frenchman and he would relay it to the rest of the expedition.

Another thing Europeans (and presumably non European nations, but I can't think of an example) did was to take someone from the local population (often by force) and train them to speak the right language. A famous example of this would be La Malinche, a Nahua woman who acted as a translator for Cortez.

Also, in a lot of cases there would be a lot of people not really affiliated with governments from different countries intermingling with native populations before any sort of formal contact was established. Trappers, traders and priests were often the first folks to interact with a population and could quite often get a handle on the local language and act as interpreters when more official contact began.