You have asked a difficult question, in part because it is hard to find a window into the mind of the 'average' person in this period. Remember that such person would be uneducated, illiterate, and unlikely to find their way into any official documentation, except perhaps a court or city record, such as they were. What we can say about them we have to construct using a good deal of hypothesis and conjecture. The best approach I can imagine off-hand is to look at the way noble society conceived of itself in this way and then ask to what extent this would have been a part of other branches of society. I don't know that I will be able to answer your question; I hope to give you something to think about.

A good starting place for trying to understanding this question is in the 8th century. Pope Zachary has just died and his successor, Pope Stephen II is meeting with the recently ascended King Pepin at St. Denis in modern-day northern France. Pepin had been the maior domus (mayor of the palace) to the king of the Merovingian dynasty, and by that time it was the maior domus who held the actual power in the kingdom, yet lacked the title. The pope's purpose for this meeting is to gain Pepin's support against the Lombards, a kingdom in northern Italy which has been challenging papal holdings. Pepin agrees, and Stephen anoints him and gives him the title "patrician of the Romans" (patricius Romanorum). As Kevin Madigan notes, "The exact meaning of this term is unclear, but it obviously was intended to augment Pepin's authority and prestige and to tie him to the dignity of the Roman imperial past--as well as to underscore his obligation to protect the Roman church." (Madigan, Medieval Christianity: A New History, p.72) Pepin will follow through on his promise, defeating the Lombards and returning/giving 22 cities over to the pope, an act often referred to as the "Donation of Pepin". This will provide the build up to the formation of the Holy Roman Empire by Charlemagne (one of Pepin's sons), relocating the center of our conceptual map of Western Europe from the Mediterranean to the north and west.

We can pause here to look closely at what took place here and ask for implications. Why did Pepin follow through on this request with such vigor? Certainly, he needed a form of legitimacy, as noted earlier. It is important to note, then, that the legitimizing language points back to the Roman Empire. This will be echoed again when Charlemagne is crowned "Emperor and Augustus" as noted by Einhard's report of the coronation (cf. Madigan, p.74-5). We can infer, if only tentatively, that there was culturally an air of importance to the Roman empire that remained in this period, enough that it could be successfully employed to grant legitimacy to a monarch. To look at another branch of life at the time, Carolingian reformers of the church were successful in conforming disparate monastic practices to the Rule of St. Benedict in large part because they presented the change as a Romanization and were viewed as Roman rulers (Madigan, p.75). While educated in their own right, monastic groups in this period were an entity distinct from the nobility, and thus we can get a hint that the appeal of Rome extended beyond merely a noble class seeking justification. Indeed, a century earlier, Augustine of Canterbury, missionary to the English isles, would write to Pope Gregory the Great asking for advice in how to conform the worship there to that of Rome, a clear indicator that 'Roman' meant 'best'; as Peter Brown notes, "The correct order of the 'ancient' Christian Church was alive in the present [8th-9th century], as the almost unconscious touchstone of all that was right and proper" (Brown, Rise of Western Christianity, p.439). In Aachen, where Charlemagne would build a permanent residence, marble and columns were imported from Rome in order to build the chapel. It is unclear that this chapel would have been seen widely outside the royal family and court, but it is not far-fetched to imagine it closer to the norm than the exception for religious architecture in the kingdom.

Yet it is difficult to say with any confidence that these ideas and visions penetrated the lower classes. Most rural churches at this time were modest (Magidan, p.82), and appealing to a Roman ideal within Christianity means little when we consider it difficult to say that the peasantry were Christianized to any meaningful extent. This is a hotly debated topic, in part because it is difficult to come across a meaningful metric for Christianization. Scholars can agree that Carolingean church reforms swept Western Europe (cf. Brown, p.443), but to the extent that this mindset trickled down, it is hard to know. I feel secure in saying that we should not assume a strong connection between high culture Christianity and low. Unfortunately, this calls into question one of the few links we could use to see whether or not the memory of Rome held by that high society found resonance within the lower.

In summary, we see by the 8th century what a strong and compelling idea the Roman Empire was within high society, and that it was indeed part of the Carolingian renaissance to connect itself to that glory. How far this went down, at least conceptually, it is difficult to say. Whether their king being called the patricius Romanorum caused the average peasants to conceive of themselves as Roman I cannot say with certainty. If I were to offer a bit of speculative judgment, the centuries between the Carolingian period and the Reformation will see a huge rise in what we today would call nationalism. There is a strong sense of regionalism such that a grievance under-girding Luther's diatribe against indulgences is that German people are being impoverished to build an Italian church (St. Peter's Basilica). How people felt in this time is difficult to say; the direction things are going, however, is away from a consolidated Roman identity and toward a more regional, nationalistic one.

Sources / Further Reading:

Medieval Christianity: A New History by Kevin Madigan (Harvard, 2015)

The Rise of Western Christianity by Peter Brown (Blackwell, 1996)

Can we perhaps expand this question to include the named parties' reactions to Otto I's coronation by the Pope? Did this actually help unify Otto's lands or was it simply seen as adding legitimacy to an already established King of Germany and Italy?

As noted elsewhere, this question is extremely difficult to answer with regards to persons outside of the elite classes we usually term "noble." I'm not aware of any sources that would give a good answer to that. I think there is also a little ambiguity here when it comes to the notion of a "legitimate heir." But there are a few points that are generally worth considering.

First, the importance of the idea of the "translatio imperii." A common medieval trope for considering history was the idea that imperial dignity had been transferred or "translated" from Rome to the German imperial lands. For the High Middle Ages, the idea is taken up most notably by Otto of Friesing in his chronicle, "The Two Cities." Although explicitly Christianizing and bearing much in common with Augustine's City of God, the parallels he draws clearly indicate the idea that dignity of Rome has passed to Germany. Similarly, Otto's "Deeds of the Emperor Frederick" owes a considerable debt to the conventions of imperial biography established by figures such as Tacitus and Plutarch (as did Einhard's Life of Charlemagne). Augustus and, even more so, Constantine were more likely to be models than Caesar, but educated emperors such as Henry IV and Frederick Barbarossa certainly would have had a self-awareness of themselves as heirs to Roman imperial dignity, and both biographies and chronicles play up the connection. Such parallels were particularly played up during the age of the Investiture Conflict as part of the emperor's claims to custodianship over the church and his divinely guaranteed place in the political order. We can see some negative evidence of this in the famous Dictatus Papae ("Sayings of the Pope") issued by Gregory VII, one of which states that the pope alone had the right to use the imperial insignia, a clear attempt to wrest away the emperor's claim to be heir to Roman dignity.

A second point regards the role of the classics in Ottonian and Salian Germany. One of the signal achievement of C. Stephen Jaeger's study of the cathedral schools of the tenth and eleventh centuries was the revelation that classical models of ethics, mores, and learning were far more important than previously thought. Stating that the education of clerics and courtiers during this period was "secular" would be going to far, but a classical, humanist ethos derived very much from Cicero and other pagan writers was central to elite culture at the Ottonian court. Whether that connection had anything to do with imperial ideology is, as far as I know, uncertain. But certainly, the educational culture of the Ottonian and Salian courts was self-consciously infused with Roman models.

A third point to consider is the geography of the empire. We tend now to think of the "Holy Roman Empire" as more or less synonymous with "Germany." But in the tenth and eleventh centuries, most of northern Italy was, in a technical sense, part of the Empire. A separate kingdom (regnum), to be sure, but still part of the Empire. On a practical level, the degree to which northern Italy actually fell under the jurisdiction of the emperor varied considerably. Generally speaking, imperial obsession with controlling Italy tended not to go well for the emperors. Still, it is worth recalling that Henry IV, although perhaps most remembered as the emperor forced to perform penance before the pope at Canossa in 1077, later invaded Italy, was crowned king of Italy, and conquered Rome in 1084. Of course, many of the Italian cities later rebelled against him (along with his son). But in Henry's eyes, he had a legitimate claim to be the ruler of Italy and Rome itself.

On several levels then, we can say that the social elites of the eleventh-century empire would have been aware of the idea of the "German Empire" as the heir to the classical Roman Empire. They were sufficiently invested in the connection to cultivate it in narrative texts of the day. It was a politically useful claim for the Emperor during the Investiture Conflict, and a prestigious claim for the German and Italian nobility generally. I have no idea how far this self-awareness would have penetrated into German society more generally. Certainly, there would have been regional differences in the Holy Roman Empire. This is a bit speculative, but it seems likely that someone in Lombardy would have been more likely to link their own past to the Roman Empire than someone in Saxony.