There are a few misconceptions here that I have to address first. Firstly, as the empire fell in 1453 (or in 1204 if you consider the Sack of Constantinople to be a strong enough break in continuity), I don't think there's any reason to say that there was a decline from Justinian onwards, certainly not a quick one! If anything, the longevity of the empire suggests that it was a strong and vigorous power for much of that time. There were for instance notable successes in the eighth century, the late tenth/early eleventh century, and in the late eleventh/early twelfth century, all of which instead indicates that the empire was as strong as ever. For the precise details of these events, you will have to ask someone else (probably /u/Ambarenya), as I can only talk about the reign of Justinian and the major crises of the seventh century, but as we shall see, even then there were still periods when the empire looked like it was going to recover.

The Romans under Justinian did achieve several major successes, but his reign also saw many disasters. The plague struck in the 540s, numerous conspiracies were recorded, whilst the wars against Persia and the Ostrogoths lasted for decades. By the time of his death, Justinian was certainly not ruling an empire experiencing a golden age; see my answer here for more details. The rest of the sixth century was also pretty bad, since the war with Persia was renewed, whilst Lombard/Frankish/Slavic/Visigothic/Avar incursions into the empire threatened all the provinces from between Spain and the Balkans. There were some notable successes, such as Emperor Maurice's peace with Persia and his reconsolidation of Roman territories in the Balkans in the 590s, but financially the empire was still in dire shape. This culminated in the revolt of the usurper Phocas, who seized the throne in 602, in turn provoking a war with Persia in 603 (the Persian Shah Khusro II had been placed on his throne by Maurice and used Maurice's downfall as an excuse to attack the empire in order to 'avenge' his benefactor). The first decade of war actually wasn't too bad, but in 608 North Africa, perhaps the only province untroubled by war for the last few decades and a hugely prosperous one at that, revolted. This was when things started to go really badly for the Romans.

The governor of North Africa's son, Heraclius, led a naval attack on Constantinople whilst his partisans attacked Egypt, which caused a brutal civil war in Egypt and massive riots in the Levantine cities. Phocas was executed by Heraclius and the new usurper became the emperor in 610, but the damage was done. The military in the east was no doubt divided by this civil war and soon the Persians were making huge gains - Jerusalem fell in 614 (which also led to the loss of True Cross!) and even more dramatically Egypt fell in 619. These were the crisis years and Heraclius resorted to melting down church silver and the cancellation of the traditional grain dole to make ends meet. The emperor personally began to campaign against the Persians in 622 and thanks to an alliance with the Turks he managed to snatch victory from the jaws of defeat, so much so that by 628 the Persian court was begging for peace. This led to a brief revival in Roman fortunes, a fact most clearly demonstrated in Heraclius' restoration of the True Cross to Jerusalem in 630. This was followed by renewed attempts to reunite the empire in religious terms by clamping down on the Jews and through the attempted reunification of the miaphysite/Nestorian Christians with the Chalcedonian Church (the doctrinal position backed by the emperor). Peace was now desperately needed for the empire to restore its economic and military situation.

Again however the empire was not so lucky. Whilst it was still recovering, the ever-present Arab raiders began to escalate their attacks on the empire and in the 630s made several astonishing gains in Syria and Palestine. By 641, such was their military prowess that the Romans effectively agreed in a treaty to hand over Egypt to the Arabs by 642. For reasons why the Arab conquests so successful, check out my answer here, but from the Roman perspective it was of course an utter disaster. Heraclius' reputation was irretrievably tarnished, whilst the loss of the Holy Land (again) led to calls for the inhabitants of the empire to repent their sins. By 649 the Arabs had even acquired a navy, which they used to attack Cyprus and later Sicily. Not surprisingly, an expensive peace was at this point bought by Emperor Constans II in order to reconsolidate what the empire still had. His island possessions were strengthened and exploited more for taxation, whilst rebellious Italians and Armenians were brought to heel.

This in my opinion led to the empire's triumph in 654. Despite a massive naval defeat for the Romans at the Battle of Phoenix earlier that year, an Arab siege of Constantinople was thwarted (largely thanks to a storm that destroyed the Arab navy, but still). Roman gains quickly followed, particularly in Armenia where the Arabs suffered another major defeat in 655. The First Arab Civil War began the next year, meaning that Constans was now in an even better position; one source even reported that more than 10,000 Arab soldiers in Egypt embraced Christianity at one point, though this is no doubt an exaggeration and is merely an indication of a Roman attempt to cause sedition in Egypt. In a very real sense, the Roman Empire reversed its decline between 654-60, so even amidst a world crisis the empire was still able to strike back at its foes.

Soon afterwards though the Arabs were reinvigorated by the rise of Mu'awiyah, the first Umayyad caliph, in 661 and war with the empire escalated once more. By 668, the Arabs were even in a position to besiege Constantinople again. Due to his failure to prevent this and the simmering anger from his courtiers over various other internal issues, Constans was assassinated in Sicily, an act of treachery that can perhaps be linked to his son, the future Constantine IV. Though the siege of the capital was not successful, the new emperor Constantine IV had to sign a humiliating peace treaty with the caliph in order to free up forces to confront an usurper in Sicily. After his success there however Constantine was able to win several major victories against the Arabs and eventually he was in a secure enough position to repudiate monotheletism in 680/1 (it's a very complicated issue that's not really relevant here, so just see Constantine's move here as a sign of his strength as he was now able to ignore pro-monothelete elements within his government).

Further successes can be identified in the next few decades, for example his son Justinian II was able to take back Roman North Africa from the Arabs, but I think I've written enough of the narrative for you to get a sense of how even in the seventh century the empire was able to recover its strength, as the empire was doing relatively well in 622-32, 654-60 and c.675-681. There were dramatic losses and by the end of the century the empire controlled only Anatolia, Constantinople, coastal parts of Greece, various islands and roughly half of Italy, but that did not mean that the empire was not able to adapt to its new circumstances. Heraclius, Constans II and Constantine IV were all capable men in their own right and if they had ruled in better times, their ability no doubt would have made more of an impact. But unfortunately they faced men who were just as adept at politics and war as they were, such as Khusro II, Muhammad and Mu'awiyah.

Perhaps more importantly, the empire still had not recovered from the cycles of the plague from the previous century, nor had it secured its frontiers to the north and the west, so there was always a constant drain on the empire's resources even when it was doing quite well in the east. Still, the empire was in no danger of falling, except perhaps for a few perilous days during the 626, 654 and 668 sieges of Constantinople. The empire would endure for another eight hundred years as it adapted to its new circumstances, eventually achieving new heights of military power under Basil II, cultural revival under Constantine VII and diplomatic success under the early Komnenians, so in no way, shape or form was the empire only declining from the time of Justinian onwards. It was certainly much smaller than before, but imposing a narrative of decline on the empire's history would be to overlook how vigorous the Roman state remained and how exciting Roman history became as it entered the Middle Ages.