I can try to provide an answer for the very early Middle Ages (6th century), which is an interesting period in many ways, which provides us with a good starting point. The best-known area in the West at that time is probably Gaul. There is every indication that its cities were quite cosmopolitan. First of all, we have extensive, if not very precise, evidence of the existence Jewish communities. Though they had been living there for a quite a while, they would still have been felt as a “foreign” presence: they had their own cults, a community that was cohesive and their own language. Important cities apparently all had their Jewish community (we get mentions in Orléans, Arles, Clermont-Ferrand, Marseille…) The same thing can be said of “Syrian” communities mentioned in the Books of History of Gregory of Tours, whose role was very probably commercial in nature. Over time, the composition of these communities can change (e.g. the replacement of Eastern traders by Italian ones), but the basic archetype of close-knits communities that retained their cultural integrity was probably quite common. It did not prevent them from playing a role in the political life: Gregory of Tours reports the existence of a Jewish advisor of Chilperic, a Merovingian king, or the accession of a Greek merchant to the bishopric of Paris. But this participation did not go in hand with dissolution in the surrounding community, even if it may have happened over time.

A specific early medieval problem is the question of Germanic communities. It seems that people knew, at least in the 6th and 7th century, if they were from a Germanic or a Roman background, even though many indicators of identity (e.g. onomastics) were already blurring. It is also clear that some communities could retain their identity for quite a while. A good example of this phenomenon is the report by Gregory of a distinct Taifal (Iranian steppe nomads) community; they were settled there nearly 200 years before. So it is probable that if you lived in Poitiers, near this military colony, you would see Taifals from time to time, even it is unclear how much they retained of their original ethnicity (in terms of physical but also cultural features). We are also told by Procopius that in Ostrogothic Italy, the Rugii, a Germanic people that had been reduced to a secondary role due to several defeats, not only maintained a sense of common identity but deliberately avoided to marry outside of their people. It is therefore clear that the inhabitant of early medieval cities would have been confronted to “Germans” (as a part of the retinue of the count, as a garrison…) which would themselves have come from different backgrounds. One of Gregory of Tours' hagiographic writings reports the presence of a Thuringian (eastern Germany) in the entourage of a count: undoubtedly, for local inhabitants, that would have been quite foreign. The same kind of points can be made, for instance, about Celtic immigrants in (French) Britanny. Regular Romans from Nantes would probably have been quite amazed by their presence, or at least would have perceived them as a distinctive population.

Religious life also drained people from various horizons, especially famous shrines. In the East, several hagiographic compendia report cures of travellers coming from exotic countries. Similarly, embassies could be a good occasion of seeing foreigners; people from Constantinople probably saw Persians, Saracens on a regular basis, though they were not permanent settlers. Religious life also brought by travelling saints but also foreign clerics — a Western Frankish cleric settling in Saxony in the Carolingian era would probably have been considered as quite foreign. Later on in the Middle Ages, the universities, which were a clercial institution, turned these occasional transfers into a massive industry. The southern part of Paris in the 13th century would have been a patchwork in this regard (even if clerics were supposed to speak Latin, the mere existence of “nations”, groups of students from a given regional background, strongly suggest that they spoke vernacular for a part of their time).

Another question worth considering, finally, is “at which point a place becomes cosmopolitan?” For instance, we would probably not consider that seeing a Syrian in Alexandria of in Cairo is particularly strange, or out of place. However, when refugees from the sack of Jerusalem (614) came to Egypt, they would probably have been considered as complete strangers. In some places, we can even wonder if someone coming from another region (say, 500 km away) would have not been taking part in the “cosmopolitan” feeling of cities.

It varied tremendously depending where and when you are talking about. I'm going to restrict my answer to the period 1000-1400 (High Middle Ages) because that's the period we have the best records for.

First of all, it's important to note that modern notions of nationality and ethnicity did not yet exist.

In general, cities that were either a political capital or on a coast or a navigable river were more likely to be larger and have a large community of foreign merchants or craftsmen. Constantinople is probably the foremost example of all of these traits -- it was a political capital, and also sat astride a major maritime trade route.

Paris was a major poltical and cultural center, but not a major commcercial center, so it's population only grew larger and more diverse in the mid-1200s onward, as the early French Kingdom accumulated powers and wealth.