I can’t speak to what the “average” person would have thought. Generally speaking we don’t tend to have sources that tell us what “average” people thought about more or less anything in the Middle Ages, and the rare exceptions (things like inquisition depositions) tend to be rather problematic in interpretation.There are a couple broader comments that I think might usefully contextualise how to think about this though.

Interpreting the term “moor” in the Middle Ages is rather problematic. Very broadly speaking, it is not unreasonable to assume that it means Islamic person. We certainly find the latin “Maurus” used synonymously with “Saracenus”, for example in the early 11th century chronicle of Adémar de Chabannes uses the combination “Mauri et Sarraceni” frequently as a sort of set phrase for discussing conflicts with Islamic powers from the 9th century to his own time. Though he also recognises a different in meaning here, as when in ~1018 a group of Cordoban “Mauri” are defeated by the residents of Narbonne and a group of 20 are captured and sold to the Abbey of St Martial in Limoges. Adémar notes (perhaps somewhat derisively) of these “mauri” that:

Their speech was nothing like that of the “Sarraceni”, but they seem to yap, speaking in the manner of puppies.

Loquela eorum nequaquam erat Sarracenisca, sed more catulorum loquentes, glatire videbantur (Chronicon, 3.52)

This highlights a second line of influence in the understanding of “Maurus” which comes from the classical ethnographic tradition, according to which the ‘Mauri’ are inhabitants of Mauretania – sort of an equivalent to Ethiopia but in the West (though the geographical tradition is considerably more complicated than this). The most important source for this in the Middle Ages is Isidore of Seville, who gives two origins of the term ‘Maurus’:

On the other hand, the Medes mingled with those Libyans who lived closest to Spain. Little by little the Libyans altered the name of these people, in their barbarous tongue calling the Medes ‘Moors’ (Maurus), although the Moors are named by the Greeks for their color, for the Greeks call black μαῦρον (i.e. ἀμαυρός, “dark”), and indeed, blasted by blistering heat, they have a countenance of a dark color. (Etymologies, 9.2.122; there is also a corresponding etymology of Mauretania as coming from the Greek for ‘black’ in 14.5.10.)

Thus equally, about a hundred years later, Guibert of Nogent describes a certain “Ethiopian” in the service of the Bishop of Laon, who is referred to latter in the text by the Bishop as “Joannes maurus meus” (John, my moor).

The usage only gets more complicated when we get into the vernacular. The traditional story is that in English sources don’t really use “Moor” until the later 15th century, but a major new article on the subject from a few months ago has just argued that we should see a growth in the use of this term as a linguistic marker for Arabic or pseudo-Arabic on textiles and wares from around 1300. So the long and short of this is that the terminology is complicated and frankly it can be difficult even to tell if something like a name that looks like “MaurXXX” or “MoorXX” is indeed a reference to Moors or just a name like Maur or Maurice.

Back to the question of interaction, there is little evidence of communities of people of color in England or broadly in northern Europe (although more on this below) around the thirteenth century of the sort we do find in the South. (The most notable example is Lucera in Apulia, a city of ~20,000 Muslims founded in the mid-13th century by Frederick II as a place to send the Muslims he exiled from Sicily.) But it is hardly implausible to believe that “regular” people might have seen or intereacted with people of colour. Setting asside the fact that people did in fact travel in the Middle Ages, either to southern Europe or indeed the Holy Land, we do find individual people of colour popping up in our sources for Northern Europe. We saw this already with the Bishop of Laon’s Ethiopian, and likewise Ademar explains that the Abbot who received these to “Mauri” kept two and gave the rest to ‘important pilgrims’ from many regions (‘ceteros divisit per principes peregrinos qui de partibus diversis Lemovicam convenerant’). But the most significant examples again relate to Frederick II, who made particular use of Africans as subjects both iconographically and directly as courtiers and members of his retinue. (Indeed, a certain Johannes Maurus was a high ranking administrator in his court.)

There are two relevant things here that might give us some indication about perceptions here. First we have various accounts of Fredericks retinue, a chronicle probably from Eberbach describes Frederick’s retinue from a 1235 journey to Swabia:

[The emperor] proceeded in great glory with numerous carriages laden with gold and silver, byssus and purple, gems and costly vessels, with camels, mules as well as dromedaries, with many Saracens, and with Ethiopians having knowledge of rare skills accompanying apes and leopards and serving as guards bringing along money and treasure. (trans. Paul Kaplain, ‘Black Africans in Hohenstaufen Iconography’, Gesta 26.1 (1987), 32)

But, as Kaplain notes, similar accounts of Frederick’s retinue tend to focus more on the exotic animals than the people. This might perhaps suggest that seeing people of colour was not as noteworthy, but we proably shouldn’t try to read too deeply into so scant a point. The more interesting result of Frederick’s prominent association with African servants/courtiers that Kaplain highlights is another anecdote from 1283, where a person turned up in Cologne claiming to be the Emperor Frederick. Now this doesn’t seem to have taken especially seriously, but as the Middle High German Steirische Reimchronik says that he brought three ‘Moors’ (drî môren l. 32242) with him as servants, one of whom was supposed to distribute treasure. Again I have no idea where these people came from, but it was apparently possible to find three "Moors" and bring them to the north of the Rhine around the 1280s.

Back to the point about communities of people of colour in northern Europe, there is one very interesting possible, albeit small counterexample here, that I’m aware of. While on Crusade in the mid-13th century, Louis IX brought a number of converts from Islam and settled them back in France. William Chester Jordan has done a very detailed study of the fiscal records for this group and finds at least 166 households were resettled in various places around northern France (to give you a sense of the numbers he notes 25 around Bourges, 14 in Laon, 11 in Noyon, 25 in Orléan, 17 in Amien and 56 in Touraine although this is not a comprehensive list; William Chester Jordan, The Apple of His Eye (2019), 98). He estimates that once we account for further members of the households that it would mean 498 immigrats. Beyond these being converst from Islam, I can’t say what these people where these people came from or what they might have looked like. But I think this is a good cautionary example to highlight how people of colour are not always transparent in the sources, and that we should be careful about letting our pre-theoretical assumptions about Medieval Europe guide us.

But as to England. It is hard to say what the response would have been. We again find people popping up now and again. For example, Christopher Tyerman notes two anecdotes about “Saracens” turning up in England during the reign of Henry III, one in 1238 and one in 1259, who apparently had them arrested as Muslim spies. (Whether Henry’s paranoia reflects a broader attitude is very difficult to say. ) The latter case was apparently “an Ethiopian called Bartholomew “sometime a Saracen”’ (Tyerman, England and the Crusades, 1095-1588, 95). What are we to make of Bartholomew? Probably not a whole lot. But from the name and description he sounds like a convert, although we certainly can’t conclude this from the name alone (cf. our Laonnois Ethiopian, John). Could he have come in by similar routes as Louis’s converts? Possibly, but the fact that he was apprehended by the crown suggests that he didn’t immigrate with knowledge of the crown. This would seem to imply that he was getting along just fine with some “regular” people on his own.