The differences in terms of legal and political structure are pretty minor, and there was a certain amount of cultural continuity. The key differences lie in language, and how they each conceived their ethnic identity.

Language should be obvious. Latin was the lingua franca in the West; Greek in the East. After Rome fell the eastern empire tried adopting Latin for a while but soon dropped it because it was obviously pointless.

Ethnic identity is more complicated. Prior to the fall of Rome, we have no indication that people in the east thought of themselves as "Romans". They were under the power of Rome (at least until the split); they had Roman citizenship; but they never ever called themselves Romans. It's possible to see romanization in some respects: people routinely took Roman-style names, even though they spoke Greek; some people would read Vergil in the east, even in Latin. But while both West and East were standing, we regularly see elites in the east going to some effort to categorize themselves as precisely *non-*Roman: often, they instead laid claim to an elite Hellenic identity that was predicated on being "educated" (their term, not mine) and sophisticated, rather than being politically powerful.

This changed the moment Rome fell. With the western empire gone, a "Roman" ethnic identity was suddenly up for grabs. People who previously had no chance of claiming to be part of the hellenized elite found that they could claim to be "Roman". And that's exactly when we see the easterners start to call themselves "Romans".

Ethnic identity isn't the whole story of course, but it's an important part. In purely legal and political terms the Byzantine Empire was a continuation of Rome. But in other ways, their claim of being "Roman" was a more artificial gesture: it worked in pretty much the same way that the Holy Roman Empire claimed to be "Roman" a few centuries later.

Short answer: What people call the Byzantine empire or Byzantium is the Eastern Roman empire, nothing less, nothing more.

Longer answer: the Roman empire was very large and difficult to control efficiently, so at some point people decided to "divide" it in different areas and have several co-Emperors, one to rule each part. Eventually the most practical division was in two halves: the Western one, with Rome as capital, and which spoke Latin; and the Eastern one, with the city of Byzantium renamed Constantinople (nowadays Istanbul) as capital, and which mainly spoke Greek (Greek had been the international language of that region for hundreds and hundreds of years). Constantinople progressively grew as rich and powerful as Rome thanks to its very strategic location. At first the two regions were supposed to be parts of the same Empire, but in practice they were independent from each other. With time the political division worsened the cultural one, and the Eastern empire became more and more Greek, but it never ceased to call itself Roman.

Meanwhile, as you know, the Western empire faced great invasions, withered and eventually fell, replaced by many small "barbaric" kingdoms. It left the Eastern as only remnant of the Roman empire. Under the reign of Justinian, it actually tried and manage to reconquer the city of Rome and large parts of the West, but soon after lost them again. So it just recentred on Constantinople and eventually abandonned Latin. Officially it remained the Roman empire, but it arguably didn't have much in common anymore especially since it didn't even include Rome.

The question of whether or not Byzantium was the Roman empire became very important in the Middle Ages, after Charlemagne was crowned Western Roman emperor by the Pope. Later Germany became the Holy Roman empire (not just Western, but the only real, legitimate one, as the other was apparently not holy), and as the Catholic and Orthodox churches split, each could back its own real Roman empire and reject the other as a fake. That's the reason why in the Western tradition we usually consider Byzantium as an entirely different, Greek empire, rather than as the Eastern Roman one.