The rest of the Empire was left intact, I think. Why couldn't the last Emperor rally the armies and population and retake the city from the foreign invaders?
In order to understand the dissolution of the Byzantine Empire following the Sack of Constantinople in AD 1204, we must first look to the Komnenian dynasty, specifically, the crucial period of time starting during the latter years of the reign of Manuel I Komnenos, and leading into the fall of the Komnenian dynasty at the hands of the tyrant Andronikos Komnenos in AD 1185. However, in simplest terms, the answer boils down to the fact that after the death of Manuel I Komnenos, there was really no one competent enough to lead the Empire against the numerous internal and external pressures that befell it. Byzantium's leaders by and large chose to ignore the Empire's dire problems until it was too late. Let's examine a few of the causes, whilst exploring the narrative.
To begin, by the late 12th Century, it became increasingly apparent to the Byzantines that the Latin West was catching up and exceeding the Empire in many realms, especially in the military aspects. In the Byzantine sources of the period, you'll start hearing about "the irresistible charge of the Latins", referring to their full-speed charges with a couched lance, something that the Byzantines were hesitant to fully adopt into their military doctrine. Although on paper, Byzantium still maintained a more organized military than its Western counterparts, it seems that maintenance and organization became increasingly more laxed as time went on, especially following the death of John II Komnenos. John's successor, Manuel I Komnenos, was said to be an incredible individual fighter, but it seems that his natural martial gifts made him hubristic, blinding his understanding of the importance of organization and overarching strategic command. The failure at the Battle of Myriokephalon in AD 1176, and several other military failures (such as Manuel's failed two-pronged invasion of Egypt, or the invasion of Italy) can potentially be attributed to this same personality flaw. Manuel was also heavily inspired by Western style combat, and often integrated Western units and mercenaries into his army, which may have been an underlying cause of the seemingly weakened effectiveness of the Byzantine military during his reign. In regards to Manuel, it is usually accepted by historians that, as Emperor, he wielded and exerted great power (the full might of the Komnenian dynasty), but he was unsure of how to use it effectively to fully achieve his lofty goals (the unification of the East and West against Islam and the revival of the old Roman Empire). Otherwise, if we reject this typical doctrine of thought, we are forced to make the less savory argument that Byzantium during Manuel I Komnenos' reign was simply exceptionally unlucky.
Anyways, during Manuel's reign, Byzantium also became increasingly more economically and militarily threatened by the Italian states, most notably Venice. The Venetians had gained many trading concessions during the reign of Alexios I Komnenos (Manuel's grandfather), who needed to open up the Empire to the West to quickly solve dire economic and military problems brought on by his incompetent predecessors. To Alexios, the West was Byzantium's initial salvation. But that salvation of course came at a great cost. Alexios must have swiftly realized when massive Crusading armies arrived at his doorstep, that the safety of the Empire would be compromised if the Byzantines were unable to uphold their image of power and grandeur. And he was exactly right, because Crusading armies would continually test Byzantine military authority throughout the reign of the Komnenoi. However, by Manuel's time, Byzantium was strong enough to deter most Western powers from inciting a Byzantine military response. However, problems started to arise in the later years of Manuel's reign, when it had become shown that despite the Empire's significant military reach, it was unable to seize and hold the territories desired by its Emperor. This, coupled with Manuel's generous personality, his zealous belief in astrology and the occult, and the resultant arbitrary decision-making, meant that the Italians were willing to try and bend the rules. And that they did. Following a number of raids between rival Venetian and Genoese factions within Constantinople's Latin Quarter, Manuel banished all Venetians and broke diplomatic relations, inciting a military response from Venice, which fortunately did not significantly harm the Empire in the short term. However, the long-term effects of this falling out with Venice would be disastrous. Byzantium's status as the foremost Mediterranean power was further challenged with Manuel's defeat at Myriokephalon in AD 1176.
With Manuel's death in AD 1180 (stated to be a swift decline in health brought on by brooding over his misfortunes - having victory repeatedly snatched away from him throughout his life), the Empire entered a period of great uncertainty. Manuel's son, Alexios II Komnenos, was only 11 years old when his father passed, and was not yet fit to rule. As Empress, Alexios' mother Maria of Antioch (a Crusader princess, Manuel's second wife) was declared as regent. Naturally, this caused a lot of controversy, as indigenous Byzantines felt that the Empress acted specifically in the interests of Western Latins, enacting laws to promote the growth of Western culture and authority in Byzantion. From the sources, Byzantines overall disapproved of her leadership (as well as stated 'scandalous activities' in the Imperial palace). Observing these events, Andronikos Komnenos, the 'Black Sheep' of the Komnenos family (an old man by this time, who took many paramours over the years, including his own niece, and gallavanted across the Middle East as a rogue after he was banished by Emperor Manuel) saw the weakness in the position of the dynasty, and seized the opportunity to not only take the throne of the Empire for himself, but also inflict damage on his hated, self-righteous dead nephew Manuel for past imprisonments and shame. In a lightning quick move, Andronikos (a fervent and well-known opponent of coexistence with the Latins) assembled an army of Turks and marched to the Gates of Constantinople mostly unopposed. After a brief exchange, the Commander of the Guard virtually welcomed Andronikos into the capital, and a massacre of epic proportions was enacted against the Latins residing in Constantinople, some say on the orders of Andronikos himself. This Massacre of the Latins became just another black mark against the Byzantines, whose reputation had never really recovered after Manuel's banishment of the Venetians in AD 1171. The Normans under the command of William II invaded shortly thereafter, perhaps in response to the massacre of Latin Christians, but also due to the very apparent instability of Empire (thus, the Normans taking a similar action to the Invasion of Robert Guiscard nearly 100 years before). This attack, unlike the Guiscard's, succeeding in cutting a large swath of destruction through Greece, and even resulted in the capture the Second City of the Empire, Thessaloniki. All the while, Andronikos, while providing some visual appearance that he was attempting to repulse the invasion had no real means to do so. Refusing any form of sane counsel, his initially popularity brought on by eliminating Latins dwindled swiftly, especially after he started persecuting/executing large groups of his own countrymen in bouts of rage and tyranny. Though he was seen as a somewhat effective leader (despite his age, and debauchery, he was said to have had significant charm and aura), a large revolt incited by the future Emperor Isaac Angelos in AD 1185 turned the tide, resulting in Andronikos' gruesome death at the hands of his own people.