The short but disappointing answer is that we do not know. The Periplus of the Erythraean Sea--a sort of mercantile guide to the voyage, and probably the most useful single source--does not provide much help here. Probably the closest mention is that the city of Muza (modern Mocha in Yemen) sent ships to Azania captained by men who knew the languages (16). Beyond that the references are even less helpful--Pliny mentions that the man who first made contact with Sri Lanka, Annius Plocamus, learned the language over six months, and he also deploys the topos of "silent trade" seen in Herodotus. This does not really get you very much except that there were, in fact, multiple languages and that how to navigate that is something that was on the ancients' mind, neither of which is particularly piercing insight.

Beyond that we can say there was a very diverse language environment, on the island of Socotra for example there is attestation of Greek, Palmyrene, and Indian languages, among others. Likewise many languages have been found in the Egyptian ports on the Red Sea, and there is evidence of Greek merchants resident in India. But there is not much we can say about the development of a lingua franca or trade language--on Socotra for example, brahmi is predominant, but given the ritualized context (cave inscriptions) this does not necessarily mean business was done in Pali.

So the best answer is that we do not know what sort of trade language, if any, developed, or what language was customary to speak in different situations, or if people just kind of went along to get along.

Worth flagging a potential note of terminological confusion: the term often translated as "merchant" is "naukleros" which actually would more commonly refer to the owner of the cargo or ship, and thus the naukleros might have been staying in Alexandria.