Several points here.

1. It wasn't called Eastern Roman Empire, it was called the Roman Empire. (Imperium Romanum, Βασιλεία Ῥωμαίων). Eastern Roman empire ceased to exist in 476, when the last Western Roman emperor was deposed and Eastern Roman emperor was then sole Roman Emperor. Also, historians consider the first proper Byzantine emperor Constantine the Great and he was an emperor of the whole Roman Empire.
2. It can be said that three core Byzantine Empire principles are Roman state, Orthodox Christianity and hellenistic cultural legacy. (+greek language). So in order to see where are "Roman vibes" there we need to examine those three principles in relation to "Roman".
3. It's really difficult to define what "Roman" is. Roman identity and Roman state changed drastically over time. People often point out how Byzantine Empire was much more hellenized then ancient Roman Empire. But hellenistic culture and greek civilization heavily infulenced Romans even from the times of mid-Republic (remmember how Cato the Elder was opposed greek influence on Romans, and that was happening in the early II century BC). So, during Roman Empire, greek culture was incorporated into Roman identity very well and there is no need to differentiate it from Roman culture (it is after all called a Greco-Roman civilization). Greek language was also the only official language besides Latin in the Roman Empire and it was especially present in intellectual circles.
4. In the sense of state and a socio-economic system Byzantine Empire started during the Late Roman System (as put in practice by emperors Diocletian and Constantine during 284-337). This system existed in the Byzantine Empire up until the VII century AD, when it started to change. In VII century AD themata system started to develop, which will be base for the ressurected Byzantine power in IX-XI century AD. But even that system was abandoned after the XI century and Byzantine system again drastically changed. So Byzantine system changed over time, but that was also the case for the Ancient Rome. Early Roman Republic was different from Late Republic and Early Empire was different from Late Empire. So we can't really say that Byzantium lost the "Roman vibe" with the change of a whole system. Precisely that will to chanhe and adapt is most Roman thing to do.
5. Christianity also doesn't differentiate Byzantine Empire from Roman Empire. Not only Roman Empire incorporated Christianity but also Christianity incorporated Roman Empire itself. It's important to note how Christianity influenced idea and ideology of a Roman Empire. That kind of thinking (Heavenly Empire in the sky - only one Christian Roman Empire on Earth) influenced not only Byzantine Empire but also other medieval European states.

So, to sum it up - in almost all aspects of political, social and economic life Byzantine Empire drastically changed and it was different from Roman Empire in that manner. But, Roman society and state also changed over time, so it is really difficult to judge Byzantine Empire in that manner. Most important aspect of a "Roman vibe" is ideological, closely connected to Christianity. For almost all of it's existence, Byzantine Empire kept that.

Suggested reading: George Ostrogorsky, History of the Byzantine State, Oxford: Basil Blackwell 1956. (Little bit outdated because it's old but good synthesis nevertheless, also deals a lot with that idea of the Roman Empire).