It is important to realize that the Roman Empire went through a great deal of change. The Empire in the west lasted for 500 years. The Byzantines saw no differentiation between the Roman Empire begun by Augustus and that of their own day. Obviously there were reforms and style changes, but it was still the same Rome. Artistically/architecturally, they still had the same foundational knowledge. Art began to look different, however you can clearly see the Roman in their art right up until the end. They were a highly conservative culture as well, something which the Romans historically were, too. The emperors still had the titles "Caesar" and "Augustus" and people still attended chariot races. Philosophically, they still had the great classical tradition both of the Greeks as well as of the Romans. Latin was the court language until the 620s AD, but it was still used on coins after that change was made. The reason Byzantium seems to very different from the Rome of Augustus was that the "Roman Empire" that Byzantium was continuing was the late Roman empire, where monarchy was a more defined and set in stone thing, whereas in Augustus' time he had to be a bit more creative about it. Augustus' empire was more or less a restored republic. The Byzantine Empire was a republic no more. The Emperor even called himself "Basileus Romaion" which means "king of the Romans" (Basileus is an interesting word to translate because it refers to a monarch. Emperor is included in that.) So the Byzantines would have though of themselves as thoroughly and properly Roman, just like Augustus, in a similar way to how Americans think of themselves as Americans, although the US looks quite different than it did during the time of George Washington.

In most aspects, they absolutely were different. Christianity was a fundamental aspect of the Byzantine Empire, you can't separate the two, while the Roman Empire, although it had its religious persecutions, was relatively tolerant of non-state religions. Emperors during the Principate (roughly from Augustus to Diocletian), Emperors weren't supposed to be seen as "emperors," they were simply someone with a collection of titles, meaning that he wasn't the law himself but was subject of the law. The senate, in theory, had the power to choose the emperor upon his death. The Emperor was expected, due to his title as First Citizen, to be accessible to all from all walks of life, not be gregarious about their wealth (this, among many things, is one of the big things that got Nero in trouble). The Emperor himself was deified and the subject of worship as well, an they had supreme authority over religious matters.

This is in stark contrast to the Byzantine Empire, where the the government was the Emperor. The Emperor had almost unlimited authority over the state. Sure, there were people who would oppose him through intrigue and plots, but the Emperor could do as he pleased with the state. He wasn't God himself, but was his viceroy on Earth, and opposition to the Emperor was seen as opposition to the Orthodox God himself. Religion, although he was certainly incredibly influential over it (Patriarchs were overthrown all the time and you have examples like Leo the Isaurian who brought in Iconoclasm), was not his "domain," so to say, and the Orthodox/Catholic church (when it mattered) operated in a separate hierarchy. All notions of him not being a monarch were removed, he was coronated in a religious ceremony at the Hagia Sophia, and the title "Augustus" (Most revered/respected) and Caesar (A family name with no inherent meaning) were dropped for Basileus (King or Soverign in Greek) and eventually Autokrator (Self ruler, meaning nobody had power of them).

On the other hand, their were similarities. Titles that no longer meant anything persisted for a long time (Patrician, Consul, Proconsul, etc). The position of Emperor, even though it was one that was a formal one, was never really viewed in the same way. According to a personal favorite book of mine, The Byzantine Republic the Emperor was viewed as a public office, not, like it was in Western Europe, as someone who owned the land of the Kingdom. As such, he was much more prone to being overthrown and replaced in favor of someone else, and the people of the hippodrome did this themselves multiple times (or at least tried to, the proclamation of Anastasius of Emperor during the Nika riots is a good example). The Emperors were still expected to be pious, kind, and just, and cruel emperors were deposed of. Roman law, for the most part, remained (The Corpus and the Basilika were codifications and modifications of old law, not completely new productions). The Byzantines never called themselves such, the term "Byzantine Empire," is a term that was employed by later historians both as a way to delegitimize their claim to be "Roman Emperors" and simply out of practicality to differentiate between the Roman Empire and its later Eastern Half. They called themselves Romans and thought of themselves as the Roman Empire, not as Greeks (although later in its history you see some revival of Greek identity).

Source: History of the Byzantine State by George Ostrogorsky.