Greek was prevalent in the eastern part of Roman Empire even in the times of Augustus.

Latin was used as the official language in the military until the reign of Heraclius (610-641 AD) when it was replaced with Greek.

Latin was used among native population even after that in North Africa until it was taken by Arabs in 699 AD, and of course in Italy.

There are really two questions here, or better two answers.

Firstly, most of the eastern half of the Roman Empire spoke Greek as a result of Alexander's conquests, which laid the foundation for the Byzantine empire to be predominantly Greek-speaking.

However, the administration, law, and politics was conducted in Latin, because it was from the start the Roman Empire and its continuation. Indeed, Byzantine saw itself as Rome, and Roman. This only officially changes in 620 under Heraclius, who replaces not only the military usage, but governmental as well.