Both 'Eastern Roman' and 'Byzantine' are relatively late terms dating to the 16th century (when referring to the empire, anyway); the Byzantines/Eastern Romans just called themselves 'Romans' and saw their realm as the rightful continuation of what we usually just call the Roman Empire.

That said, these terms are helpful because it is easier to refer to the empire centred around Rome as the 'Roman Empire' (and, later, the 'Western Roman Empire'), and to the empire centred around Constantinople/Byzantium as the 'Byzantine Empire' (or the 'Eastern Roman Empire') to avoid confusion. Some historians refuse to use the terms out of principle, but by far most don't really mind - my university has a centre for "Classical and Byzantine" studies, for example.

It is rather confusing as the Roman Empire in its end days split and remerged in various pieces before the Western Roman Empire (centered on Ravenna by then) finally fell. So there is a time where the Empire is split in 3 pieces so there is a Western, Middle and Eastern part. That Eastern Roman Empire is not equivalent to what is commonly understood as the Eastern Roman Empire (i.e. a 2-way split).

The next problem is when the Eastern Roman Empire ends and the Byzantine Empire begins. The founding of Constantinople as the new capital in 324 AD ? Or its consecration in 330 AD ? Or the death of Constantine I in 337 AD ?

Various authors will use various choices so there is some confusion. Still, the fall of the Western Roman Empire was a historic milestone so a name change is a good decision, as the character of the 'Roman' empire was different from the old one (it was Greek for instance).