It just seems odd that well over a century passed between 476 and when Heraclius made the change. Even considering Justinians' reconquests. I mean educated Romans favored Greek over Latin since the late republican era anyways. Why didnt an emperor before Heraclius do it?
I have to begin by pointing out that the evidence for Heraclius changing the official language of the empire is minimal, as it is dependent on one of Heraclius' edict in 629, which included in his titulature this important phrase 'πιστὸς ἐν Χριστῷ βασιλεύς' (transliteration: 'pistos en Christo basileus') - 'emperor, faithful in Christ'. The gist of the argument is this: previously, official documents were written in Latin and Greek sources would have used 'autokrator' to refer to the emperor, the Greek translation of the Latin 'imperator' (a relic of the Romans' aversion to anything monarchical). 'Basileus', however, definitely had a royal connotation and is previously only used to refer to 'barbarian' kings and the emperor of Persia. As a result, when Heraclius referred to himself as a 'basileus', this was viewed as an important turning-point, as the use of this Greek word can be seen as an indication of a change in the official language of state and a wider cultural shift within the Roman state.
Unfortunately, I don't think this argument, originating in Ostrogorsky's Geschichte des byzantinischen Staates (1940), stands up to scrutiny today. For one thing, we know that Greek was already used within the imperial bureaucracy, perhaps as early as the fifth century under Theodosius II, as a praetorian prefect then allegedly issued laws in Greek. Nor was this a unique occurrence. From Christopher Kelly's excellent Ruling the Later Roman Empire (2004):
In 397 the emperors Arcadius and Honorius had allowed provincial governors to issue their judgments in Greek. In September 439, Theodosius II confirmed the validity of wills written by Roman citizens in Greek. By the turn of the fourth century, Latin was no longer the language of instruction at the empire’s most prestigious law schools in Constantinople and Berytus (modern Beirut). Legal textbooks and commentaries were written in Greek. From the mid-530s on, most of Justinian’s legislation (unless specifically directed at the Latin-speaking West) was promulgated in Greek. His law of 535 ratifying the mid-fifth-century emperor Leo’s regulations on the alienation of property by religious foundations set out the advantages of this change simply and clearly. “We decree that this law [of Leo] shall be valid in all these matters, and we confirm it. And for this reason we have promulgated and caused it to be written not in the language of our ancestors but in everyday speech, that is, in Greek, so that through ease of understanding it might become known to all.”
More evidence can be found in the sixth century. John Lydus, a bureaucrat working in Constantinople, complained bitterly that less and less officials were learning Latin - for him, knowledge of Latin was an essential part of being Roman. His contemporary, Procopius, also noted that (albeit in a text written to diminish Justinian):
He [Justinian] then appointed Junilus (a Libyan by birth) [as the quaestor], a man who had not so much as a hearsay knowledge of law, for he had not even studied it in the public schools. Although he had a knowledge of Latin, he had never had any tuition in Greek, and was unable to speak the language. Frequently, when he attempted to say a few words in Greek, he was laughed at by his own servants.
In both cases, it is clear that the use of Latin was nowhere near universal within the upper echelons of power. Of course, this did not mean that Greek was now the official language, since Latin was evidently still being used, most dramatically in Heraclius' silver coinage, which featured the Latin phrase 'Deus adiuta Romanis' - 'God, help the Romans!'.
As a result, I don't think we can definitely say that Heraclius changed the language of state during his reign - at most, he was making official informal trends. As /u/Growlinghamster pointed out in this thread, 'the use of the title “Basileus” wasn’t some radical, out of left field change that Heraclius just sprang on his unsuspecting subjects'. The same, I think, was true for the usage of Greek - we simply don't have any evidence that any emperor had deliberately changed the 'official' language (especially since the empire still possessed territories in the Latin West). To prove this, we would probably need a surviving legal text or an extensive discussion of the edict in a historical source, none of which exist to my knowledge. It is better, I think, to view the changing prominence of Latin and Greek within the Roman Empire as a gradual thing, rather than a sudden transition, since this interpretation fits better with the evidence we do have.