Well, that's a long and complicated question. I'll try and keep it simple and summarized. It's probably best to start by stating that the Roman empire was technically still a single entity - the division between east and west is a modern invention- and that "Byzantium" was really just the continuation of the empire.

But first, let's start by tackling the fall of the west, and then the fall of the east.

There have been a number of theories as to why the west fell, and scholars still disagree. Edward Gibbon was the first to truly try to explain it, and although his work is outdated now, it's still a good read. That being said, probably the best modern work is Peter Heathers' "The Fall of the Western Roman Empire." Heather goes into detail about the increasing organization of the Germanic peoples as a result of the rise of Rome, and Roman influence on their culture. He describes the collapse of the Roman ability to assimilate people emigrating into the empire in the late fourth through fifth centuries, where instead of becoming a part of Roman culture, they co-opted Roman culture to form their own identities. Probably most centrally though is his argument that the Roman loss of north Africa (modern Morocco, Algeria, Tunisia, and Libya, which were once very fertile grain and olive production centers) resulted in economic collapse and an inability to fund a professional army, causing severe military cuts which led to a spiral of decline as more territory and taxes were granted to federate peoples and they gained more power over the Roman state which had less and less income to fund the military or pay the federates. The western Roman empire effectively continued to exist in most ways through the reigns of Odoacer and Theodoric the Goth, after Romulus Augustus had been deposed, but after the Gothic wars and Justinian's reconquest, Italy was fundamentally changed.

Some of this can be debated, and rightfully so, but it's a solid explanation for why the west fell. The fall of the west can't be attributed to a single factor, but it was a combination of things ranging from Attila the Hun to Italian Aristocratic control of tax policy. I should note, however, that the "Christianity" and "Lead in the Water" theories have been thoroughly debunked.

The Eastern Roman empire took much longer, and arguably could have fallen in just as short of a timespan had certain events played out differently, but that's not possible to prove because it didn't happen. Roman decline in the east can really be separated into three phases - after Justinian's conquest through the rise of Islam (565-718), the decade after Manitzikert (1071-1081), and then the events from the sack of Constantinople through the fall of Constantinople (1204-1453).

The first phase was really triggered by the Black Plague - the first known worldwide epidemic, which is estimated to have killed about 33-50% of the population of Mediterranean Europe in 541-542 AD, with subsequent outbreaks over the next two centuries. Mass population die-off resulted in manpower deficiencies and tax problems, which forced restructuring of the empire's administrative and economic systems. The need to focus on the Sassanid frontier meant they couldn't effectively hold Italy or South Spain, and Roman control in the reconquered regions declined. Africa was held onto, however, for some time and was vital to the Empire's survival of the Islamic conquests. Islam succeeded against the Romans due to a combination of factors: namely military weakness after decades of war between the Romans and the Sassanids, along with dissatisfied populations within the empire due to religious conflicts between the church and various heresies. The Battle of Yarmouk, in which the Romans were very badly defeated trying to halt Muslim advance, is arguably considered one of the most important battles in world history. Over the next few decades the Romans had lost their primary tax and grain production center in Egypt, as well as Syria which was a major military center and also highly profitable in both trade, tax, and agricultural produce. Africa held out for almost another century - but eventually fell in 698-701. Haldon argues that without the brief periods of disunion within the Islamic world, which gave the empire breathing room to reorganizes, it wouldn't have survived either. Ultimately it was the breakdown of the Umayyads after the failed second siege of Constantinople that allowed the Romans to truly restructure themselves and enter into a "holding pattern" that lasted until about 950 AD.

From about 950ish to 1045 the Romans once again were on the rise. In this brief period most of their competitors were in disarray - the Islamic world was broken up, Europe was disunited, and although not directly relevant China was transitioning Dynasties which obviously had economic repercussions. The Romans pushed and took back most of South Italy, part of Sicily, significant chunks of Syria and Mesopotamia, most of the Balkans, and all of Armenia in this time, thanks to Basil II's policies in repressing the Anatolian aristocracy and his funding of professional armies. However his successor's policies ultimately screwed up a lot of what he did, and as a result when the Romans lost the Battle of Manitzikert, which really shouldn't have been a big deal, plotting at the court between anatolian families resulted in usurpation and the breakdown of Anatolia, which allowed it to be overrun by Turkic raiding since the Theme system had fallen apart.

The Komnenian Emperors managed to recover significant amounts of lost territory, but ultimately their efforts were undone by the Fourth crusade, which sacked Constantinople and divided up the empire. The empire didn't have the capability to recover from that point forward, not without some sort of massive reforms at least which never occurred. At that point the fall of Constantinople was a matter of time, the only thing that really made it exceptional was how long they held out.

There's of course a variety of other factors, but I can't possibly list them all. Entire books are written just on single aspects, but if you have questions on those, I can elaborate.

For further reading see: Heather, "The Fall of the Roman Empire: A New History of Rome and the Barbarians"

John Haldon: "The Empire that Would not Die: The Paradox of Byzantine Survival"

John Norwich: "A Short History of Byzantium."