They were referred to by these names and many others.

The Donation of Constantine, an 8th century fraud shows some of the terms that Franks and other Western European peoples were using: Imperator Graecorum (Emperor of the Greeks), Imperium Graecorum (Greek Empire), Graecia, (Greece), Terra Graecorum (Land of the Greeks), and Imperium Constantinopolitanus (Constantinian Empire) all show up in the document. Of course this was a fraud intentionally designed to shore up Western claims of primacy over the East. And as such it is playing up the "Greek" nature of the East.

And these and similar terms do show up in other documents of the time. But the most common term for the empire is probably the Latin word Romania (Land of Rome) compared to Roma (which was reserved for the city of Rome). And they seemed to prefer the term Imperator Romaniae (Emperor of the Land of Rome) over Imperator Romanorum (Emperor of the Romans), that was reserved for the Western Emperor.

There is a good book, Letters from the East by Malcom Barber that has a lot of letters written by crusaders back home. By my rough guesstimate the empire is almost always called Romania. The people are called either Roman or Greek... with an even split.