From my understanding, terms such as "Byzantine Empire" and "Eastern Roman Empire" are applied centuries later to describe the Roman Empire in the East. However, it is also understood that the Byzantines considered themselves the continuation of the Roman Empire. But what do they call themselves after Diocletian's Division where the west still exists? Just Roman Empire even after the division of administration? Do they have anything that distinguish themselves from the west?
The Byzantines were distinguished from the Western Romans by language and faith--they spoke Greek and were Orthodox while the Western Romans spoke Latin (later Italian) and were Catholic. In the 730's the emperor in Constantinople declared icons to be idolatrous (the origin of our word iconoclast) while Rome maintained that they were not. The two empires even came to use different scripts, with the Byzantines adopting Greek miniscule in the ninth century onward.
The Byzantines never stopped referring to themselves as Romans and retained a sentimental attachment to Rome itself even after the cultural and power center of the empire shifted to Constantinople and after the division of the empire. Justinian reconquered Rome in the 6th Century, and when Charlemagne was crowned Holy Roman Emperor in 800, the Byzantines continued to refer to him as merely "Emperor of the Franks."
We see an interesting linguistic echo of this in Turkish--the Ottomans called the Byzantine Empire Rhûm.