For early Byzantine history, you can't go wrong with reading Procopius' History of the Wars. Procopius is rightfully called the last great historian of Antiquity, for other secular histories from this period only survive in fragments or are horrible to read (avoid Theophylact Simocatta!). He lived in the sixth century and had personally experienced some of Justinian's wars, so Wars is by far the most useful source for Justinian's reign, who is perhaps the most well known Eastern Roman emperor. You can then try to read Procopius' more colourful Secret History, which damned Justinian as a demonic tyrant and contained many fanciful stories, particularly salacious ones about the Empress Theodora. Great read, but treat it cautiously as a historical source! :)

For secondary sources, you can try John Norwich's Short History of Byzantium, Judith Herrin's Byzantium: The Surprising Life of a Medieval Empire or the Oxford History of Byzantium, all books that cover the entire history of the empire and are written by excellent historians.