Did both think of themselves as the Romans or the Roman Empire?
This answer concerns only the ninth and tenth centuries.
Authors and scholars connected with the Carolingian (and later Ottonian) court typically referred to the Byzantine Empire as the Kingdom of the Greeks (and sometimes called the Byzantine emperor "the King/Emperor of Constantinople"). The Byzantines, for their part, called Charlemagne's realm the Kingdom of the Franks.
After Charlemagne's coronation as Holy Roman Emperor, his subjects (or at least his fawning court officials) dutifully called him Emperor of the Romans. The Byzantines, of course, had never ceased to call themselves Romans, and always referred to their ruler as King of the Romans (basileus romaion) or simply emperor (autokrator).
The really difficult issue was imperial titles. The Byzantines could never acknowledge a barbarian upstart as Roman emperor, and the new Holy Roman Emperors were reluctant to concede political supremacy to the Byzantines.
In diplomatic correspondence with the Byzantines, Charlemagne was always careful to refer to himself as "Emperor governing the Roman Empire" (Imperator Romanum gubernans imperium). After years of snide letters addressed to the "King of the Franks", the Byzantines consented to style Charlemagne and his successors simply as "Emperor," with no indication of what region or people they happened to be emperors of. Byzantine emperor Michael II, however, couldn't resist addressing Charlemagne's son Louis the Pious as the "glorious king of the Franks and Lombards, who is called their emperor." Eventually, it became conventional for the Byzantine and Holy Roman Emperors to address one another as "brother."
Source:
M. Anastos, Aspects of the Mind of Byzantium (Political Theory, Theology, and Ecclesiastical Relations with the See of Rome), Ashgate Publications, Variorum Collected Studies Series, 2001.
I talk more about this, for what it's worth, in my page on Carolingian Rome.