I’ve often heard that by 1453 the fall of the Byzantine Empire was inevitable and even if they had repulsed the Turks the city would have just fallen later. At what point did the Empire go from a precarious but still viable state to one that was essentially doomed?

There are many different views on this issue. For a long time, the Battle of Manzikert in 1074 was seen as the beginning of terminal decline. The Empire lost most of its Anatolian heartland to the Seljuk Turks, and was never able to reclaim it. Without the revenue and manpower those areas provided, the Byzantines were vastly weakened. However, Emperor Alexios Komnenos and his successors managed to stabilize things and regain some territory following the First Crusade two decades later, and for the next century Byzantium experienced economic prosperity and relative military and diplomatic success. Therefore, Manzikert is no longer seen as being the total cataclysm it once was thought to be.

In 1204, the soldiers of the Fourth Crusade laid siege to Constantinople on behalf of a Byzantine claimant, but this claimant was unable to pay the Crusaders what he promised, and got overthrown. His successor basically told the Crusaders to get stuffed, so they sacked the city and looted everything they could, and set up the Latin Empire and a few other petty kingdoms to rule over Greece. A Byzantine successor state in Anatolia, the Empire of Nicaea, managed to retake Constantinople and most of Greece 50 years later, and its rulers were generally recognized as being the legitimate Byzantine Emperors. They would rule until the fall of Constantinople in 1453.

In many ways, the Fourth Crusade was a death blow that simply took two centuries to take effect. The collapse of Byzantine authority allowed many of their old enemies to rise back into power, such as the Tsardom of Bulgaria, the Serbian Empire, and most importantly, the Turks of the Rum Sultanate and its successor states. Meanwhile, relations with the West had completely soured after the Crusade, with the restored Empire constantly warring with the remnants of the Crusader states in Greece. Furthermore, fifty years of foreign rule/occupation by the Crusaders hadn't done much for the economy of the region. In the end, the Byzantines wound up facing enemies on every side—a situation they had endured before—but they no longer had the resources to survive it. If the Ottomans hadn't finished the job, someone else would have.