This seems like more of a question of language than of history, but the Eastern Roman Empire is Eastern relative to the Empire as a whole. The extent of the Roman Empire doesn't conform to modern ideas about the boundaries of Europe. When the Empire was formally divided, the two praefectures of Illyricum and the Orient made up the Eastern Roman Empire, which therefore included parts of what people might now see as Eastern Europe such as the Balkans. But it also included Egypt, the Levant, Greece, and Anatolia.

That's just taking a snapshot of the theoretical divide from the Notitia Dignitatum, of course as time went on the frontiers of the Byzantine Empire changed. To re-emphasise, I think it's important to try to avoid bringing too much baggage with the term 'Eastern'. So as I said, in a purely nomenclature viewpoint, the East here is simply referring to the Eastern section of the Empire as a whole.

If you want it more explicitly described, here is a link to the Notitia Dignitatum online. The document is a text describing the offices of empire, and is divided between East and West. Amongst lists of members of a civic bureaucracy, there are also geographically situated positions, such as Praetorian Prefects, Proconsuls, etc.