So I've been reading John Julius Norwich's excellent three part history of the Byzantium. In book one The early centuries he mentions that either the death of Justinian I or Herakeles marks the transation. But I was wondering what other historians think of the devide or what other milestones there were.
For me, there is no such distinction. It was basically the same empire, so there is no reason to call them different things, though I'll admit that Byzantine is a useful term to describe the medieval phase of the empire (it is also used by some as a convenient shorthand for the Eastern Roman Empire from the time of Constantine onwards). I especially don't see any reason to mark Justinian or Heraclius' death as the turning point, since from my perspective very little had changed after their reigns. The seventh century was definitely an age of crisis and the empire contracted dramatically in this period, but there is no real reason to see the reigns of Constans II or Constantine IV as particularly different from that of Justinian - they all ruled a pan-Mediterranean empire and did so through the same institutions. There were some differences, but when they occurred is debatable. Heraclius' usage of Greek in his official titulature is often cited as evidence for how much the empire had changed, but as I explained here and here, the evidence isn't that conclusive at all. Likewise for the Theme system, something that is perhaps quintessentially Byzantine for many people, but I personally don't have a clue when it was created, since historians have suggested many different dates for its creation. As many of these changes happened gradually and incrementally, I think it would be very difficult to find a clear demarcation point between the 'Roman' and 'Byzantine' phases of the empire. 'Roman' in my opinion is as good a term as any for describing the empire.
This is of course only my view. Averil Cameron, perhaps one of the most prominent Byzantinist working in this field today, had this to say:
However much one wishes to avoid the dangers of seeming to argue for continuity, it is impossible to avoid the question of periodization in relation to Byzantium. As I have noted, several recent writers prefer to see “Byzantium” proper as beginning from ca. 600 or later, and there are good reasons why. Constantinople was formally inaugurated in AD 330, but there was not yet such an entity as “Byzantium,” distinct from the eastern Roman empire, and it remains the case that the Byzantines thought of themselves as Romans. The shock and loss of territory consequent on the Arab invasions of the seventh century also necessitated a painful adjustment. Nevertheless, adopting a later periodization risks obscuring the fact that what we call Byzantium had a long earlier history; it was not a new state formed only in the medieval period.
This is I think a reasonable summary of modern scholarship. The Byzantine Empire is 'Byzantine' because it was ruled from Byzantium, but it was still just the Roman Empire, even though it looked very different in 1453 (a result of reforms, wars and changing cultures over many centuries). Otherwise, proponents for the usage of 'Byzantine' basically cite examples of how the empire changed in the seventh century, a point that I would debate, but it's also a position accepted by many more experienced Byzantinists.