It honestly really depends on the time period you're looking at. The history of the Roman state stretches over more than 1000 years (from ca. 500 BCE to, at least, 565 CE), and for Byzantium you have to add another millennium until 1453, when the Ottomans conquered Constantinople and thus dissolved the Byzantine administration. So if you're looking for a more detailed answer, which I would be happy to provide, you'll have to narrow your question down to a more precise time span/to a specific era of Byzantine history – and, of course, a period of Roman government to compare it with.

Roughly speaking however, the Byzantine Empire featured a Roman government, a Greek culture and a Christian religion, as Ostrogorsky famously put it in his seminal work first published in 1940. And, fundamentally, he was right. There was no clear cut separating the Byzantine civilization from the Roman imperium. Byzantium, which arguably came into life in 330 CE when Constantine founded his "New Rome", was at first nothing else than a new period in the history of the Roman Empire. Therefore, the Byzantine government didn't differ much from the contemporary Roman structures, which had developed under Diocletian and Constantine.

However, these structures developed due to internal factors (most importantly the permanent separation of East and West in 395 CE) and expansion (the administration of Italy had to be greatly reformed after Justinian's partial reconquest of the west). Most importantly however, it was external pressure which forced the empire's institution to adapt – at the beginning of the 7th century, the Byzantine state gave up the century-old Roman provincial structure and adopted a form of political and military organization based on a new administrative unit: the Theme. This drastic change in the makeup of the empire is quite often regarded as the moment when the Byzantine state ceased to be Roman and became something else, something new (although it's of course important to remember that the Byzantines never stopped to consider themselves to be Romans).

To conclude, I can only recommend you Ostrogorsky's History of the Byzantine State, on which I based this post and quite a lot of my knowledge on the subject – there are, of course, quite some books that are more recent and up to date, but History has become a classic worth reading.

If you want to get more details on any question, feel free to ask.

Source: Georg Ostrogorsky, Byzantinische Geschichte 324 bis 1453, Munich 1965.