Those that we today call Byzantines refereed themselves to as Romans, in Greek "Ῥωμαῖοι". They considered themselves of 1) Greek language, culture, literature, and science, 2) Roman law and tradition, 3) Christian faith. They saw themselves as legitimate heirs to Rome.

Since the West and the East diverged in religious views and the Holy Roman Empire was founded, the westerners stopped referring to the "byzantines" as Romans because they didn't want to aknoweldge them as the heir of the Roma Empire (because they wanted the HRE to be considered as such). Yet, they could not deny that there was an emperor in Constantinople. Byzantium was rather perceived to be a corrupted continuation of ancient Greece, and was often derided as the "Empire of the Greeks" or "Kingdom of Greece". According to Halsall (1997) [Source] (http://legacy.fordham.edu/halsall/source/urban2-5vers.html), in order to consider themselves the heirs to Rome, the Franks referred to the Byzantine Empire as "Empire of the Greeks".

From the speech of Urban II calling for the crusades (in this case he was urging Christians to help the Byzantines):

The kingdom of the Greeks is now dismembered by them [the Turks] and deprived of territory so vast in extent that it can not be traversed in a march of two months. [Source] (http://legacy.fordham.edu/halsall/source/urban2-5vers.html)

Famously, bishop Liutprand of Cremona (c. 920 – 972), in a diplomatic mission in Constantinople, was thrown in jail after pope John XIII called Emperor Nicephorus "Emperor of the Greeks" [Source] (https://en.wikipedia.org/wiki/Liutprand_of_Cremona)

In summary, the answer is that you would call them "Greeks".

The term Byzantine comes form he historian Hieronymus Wolf (1516 - 1580), created in order to avoid a historical mess by calling them the Romans.