During the middle ages, northern Italy was usually under the control of Germans or Franks with the exceptions of Venice. Southern Italy (and Sicily) was under the control of various groups such as Byzantines, Normans, Muslims, etc. Why couldn't the Italians fight off invaders?
While there is a distinction to be made between economic activity and a polity's capacity to act (a prosperous society is not necessarily a militarized one) I think there are a few narrative points I'd like to straighten out in order to examine each foreign influence on the Italian peninsula on its own terms.
Regarding the Byzantine Empire's presence in Italy, I think it is difficult to speak of a Byzantine "occupation" as the constructs tied to the "Roman" Empire hadn't entirely evaporated by the end of King Theodoric's monarchy (the first entirely Italian polity since the early Roman Republic) and indeed notions of "Universal Empire" would persist in some way or other even past the Empire's evacuation of its last outposts in Italy (let's not forget, for example, the Byzantine's displeasure with Charlemagne's appropriation of the "Imperial" title). Theodoric's own success was centered around his ability to successfully collaborate with the peninsula's old institutions of imperial power.
Of course, the first Regnum Italiae did not meet a good end. Following Justinian's reassertion of direct imperial control in Italy by force of arms, the centuries of destructive conflict which followed would pit a foreign warrior-class and the old "Senatorial" aristocracy (that is to say, "Lombard Italy") against the the Eastern Empire. But that is not the whole story, as for the Greek-speaking South, the maritime cities, and a good segment of the Italian elite which was well-integrated with the Western Mediterranean, this conflict can't have been all that surprising, and possibly more similar to an umpteenth conflict internal to the empire rather than a foreign occupation. And the Lombard political organization, in spite of its incredible flaws and recurring internal conflicts, can be said to have been largely successful in its systemic conflict against the Byzantines. While there was no decisive victory whereby the Lombards could definitively oust an Empire which, at that point, was still the greatest in Europe, there is no doubt that the Empire's footholds consistently shrank as the Lombards expanded their areas of control.
Is is in the context of this destructive and unfinished war that the Carolingian Conquest and the Abbasid conquests occurred.
The story of the Carolingian conquest of Italy in the late eighth century is a dramatic one. I have rambled on about it here if you're interested. It unsurprising than once the Frankish elites constructed a massive state in Northern Europe, this christian, latinized, and highly militarized political apparatus would seize upon the flimsiest of pretexts to turn to Italy, still relatively prosperous in spite of decades of war, certainly a prestigious prize, and politically divided.
What opposition could the Lombard leadership offer? The Carolingian Franks were a united front behind a strong monarch ruling a large state with a strong centralized government, while the Italian elites only ever successfully controlled about a third of the country, with significant factions flipping between the peninsula's warring powers (Lombard and Byzantine) as their advantage dictated. The Lombard defense, given the circumstances, was more than impressive.
While in the Carolingian period we have a mess of dynastic breakdown and conflict with every succession, Carolingian Italy didn’t really suffer significant divisions or fractures beyond changes of leadership (certainly not experiencing as many divisions as the Frankish heartland along the lower Rhine). Eventually, the increasingly autonomous Italian aristocrats would contest the empty title of “Emperor” among themselves: While the Carolingian Empire had fundamentally relied on the existence of a single strong and charismatic monarch, the inability to guarantee stable transmission of power allowed local administrators who theoretically held appointed posts to instead entrench themselves as powerful regional dynasties. So Carolingian Italy too, was a society which quickly became very different from whatever modern notions of occupation we might have as modern readers.
A fading Byzantine Empire in the South of Italy, on the other hand, would see the Island of Sicily become easy pray to the Abbasid Caliphate over the course a long conflict in the ninth century. Here too, there is objectively little that the Sicilians themselves could do, caught as they were in the broader conflict between the Abbasids and Byzantines. And besides, Sicily spent less than a century under Abbasid control before being emerging as the center of one of Italy’s largest polities in the eleventh century: The Kingdom of Sicily. I also wrote about that here if you’re interested in reading more.
That same century in which Southern Italy was unified in a single state, the German Kaiser Otto unified the German and Italian monarchies in one of the more telling developments in Medieval Italian History. If Otto's objective was to reestablish the Carolingian Empire, why did he descend into Italy over the Alps instead of crossing the Rhine into France?
Setting aside the fact that Northern Italy is much smaller than France (and thus an easier target in purely logistical terms) Italy was a particularly easy target. Tellingly, Berengar of Ivrea, crowned King of Italy after a lifetime of plotting, was in a particularly weak position. There were a number of institutional quirks in Italy weakened centralized power (and thus hampered the ability to organize things like large military operations) but it was these very power structures which allowed for artistic, economic, and social development. That development, in turn, would create sustained disunity. Could this cycle have been broken? Probably not, but not for lack of trying.
An important point has to do with Italy's Roman heritage: it is not a consequence of happenstance that the center of Europe's greatest empire was a peninsula characterized by dense urbanization, high agricultural productivity, and strong internal and external trade links. This afforded the two post-Roman kingdoms of Italy (both Lombard and Carolingian) an enormous potential for internal instability: with relatively small power bases, aristocrats could easily challenge the state. On the other hand, even in times of tranquility the state could never adequately affirm itself. Let's not forget that Lombard Kingdom in particular never even consolidated control of the entire Italian peninsula, with entire corridors connecting major urban/productive areas tethered to the Eastern Empire thanks to their commercial and cultural links in addition to Constantinople's will to commit political, military, and material resources.
What were these quirky institutions of power? Urban-centric political organizations which fostered inclusiveness, enfranchisement, rule of law, and as a consequence encouraged trade and prosperity. A collaborative sort of government emerged whereby local dynasties ruled together in "Communal" governments seated in cities. And this fragmentation further drove competition, efficiency, and a drive by collaborative political institutions to facilitate productive forces to become as competitive as possible, pushing wages upwards, accelerating the economy, raising the standard of living, and creating an environment that could even support innovation for innovation's sake in fields like the arts and sciences.
An further examination of Italy under Emperor Otto, and the evolution of the relationship with Holy Roman Empire, follows below.
Here's a little explanation of mine for the origins of Italian maritime commercial wealth and its relationship with the geopolitics of Italy. Prior to the Norman Conquest of Southern Italy and Sicily, which took over a century to complete and only really got going from when the de Hauteville brothers (Roger and Robert) arrived from Normandy in the mid-11th century, Southern Italy may have been under "foreign" rule but it was very fragmented. The Heel and Toe of Italy was under the control of the Eastern Roman Empire until 1071, but much of Italy was under the control of various Lombard principalities like Benevento, Salerno, Capua and Amalfi. Naples was in a bit of a weird situation - it was a semi-independent polity which was nominally part of the Eastern Roman Empire but it relied heavily on the Pope and the Normans for protection. Now the Islamic world, though heavily interconnected through trade and commerce, was not a hostile monolith, and if anything the Islamic world needed cheap labour - there was simply too much money going round (whilst a farm with a workshop cost half a silver denier a month in rent in Pisa in 1052, a silver dinar could only buy you lunch in Cairo) and the workers of the factory towns of Fatimid Egypt, like Tinnis and Damietta, had a lot of bargaining power with merchant-manufacturers having to compete for them and, though you could employ slaves, they were still more expensive at that end of the Mediterranean (purchasing a slave in Pisa would cost 5 dinars, but a slave in Alexandria would cost 30 dinars). So the Muslims looked to Italy, which they had strong contacts with through Aghlabid Sicily, and so began to trade with Capua, Naples and Amalfi, from which they would outsource their textiles - from as early as 928, we hear of Arabs buying linen in this area and as late as the 1150s, al-Idrisi would describe Naples as the "City of Linen." In return for this, they would get gold coins from Fatimid Egypt and ceramics from Tunisia and al-Andalus, and very soon Capua, Naples and Amalfi entered into a currency union with the House of Islam (which they couldn't have done if they were part of larger more centralised empires) to lower transaction costs. These Southern Italian polities then acted as middlemen for the Italian polities further North i.e. by the 970s, Islamic pottery can be found in Pisa.
The 11th century is a period of crisis everywhere in the Mediterranean. The Caliphate of Cordoba fragments into several different princely city states known as taifas in 1031 after al-Andalus is devastated by a civil war. Aghlabid rule in Sicily breaks down, leading first to less successful Roman and then more successful Norman invasions. Fatimid Egypt experiences a great economic crisis resulting from a low Nile and the gold reserves of upper Egypt drying up in the 1060s, which leads to a terrible civil war in the 1070s that makes the caliphs essentially puppets of military vizirs. The Eastern Roman Empire loses ground to both the Normans in the West and the Seljuks in the East. The Western Reich experiences the civil wars of the investiture controversy in the 1080s and 1090s, which lead to a breakdown of imperial authority in Northern Italy and much of Germany. Finally, the First Crusade creates new Latin principalities in the Levant. This couldn't be a more better situation for the city states of Northern and Central Italy to take off. They already had a lot of autonomy to begin with, under their own Bishops. Indeed, the reason why the dispute over lay investiture between Emperor Henry IV and Pope Gregory VII came to civil war was because both needed some way of controlling the bishops, highly autonomous spiritual leaders and temporal rulers in their own right - Henry because without being able to appoint the bishops, effective imperial control over Italy would be lost, and Gregory because once he was able to appoint the bishops he could actually get in the driving seat for church reform. From the 1060s, we thus start to Pisa and Genoa develop merchant marine fleets and begin to forge commercial alliances with certain Muslim Taifas in Spain (Denia and Lerida in particular) and raid Sicily and the Balearic islands (the raid on Palermo in 1064 helped fund the construction of Pisa's great Romanesque Cathedral, began in the same year) and then towards the end of the century join in with the First Crusaders to take over the ports of the Levant, with Daimbert, Bishop of Pisa, being made the first Latin Patriarch of Jerusalem. In the first half of the 12th century, trade agreements on much more equal terms to those held by Naples, Capua and Amalfi a century and a half earlier are being made between Pisa and Genoa (who are now very much independent republican states in their own right) and Fatimid Egypt and the Eastern Roman Empire. The wealth of these basically independent Italian city republics, which are now only really nominally within the Empire, is so attractive that when Frederick Barbarossa tries to restore effective imperial control over Italy, the loot from the captured Italian cities is a big incentive for his German followers to come i.e. Otto of Freising recounts all of this in quite a lot of detail in his "The Deeds of Frederick Barbarossa."
Recommended further reading
"Italy in the Central Middle Ages" by David Abulafia
"An Economic and Social History of Medieval Europe" by S.A Epstein
"Sleepwalking into a new world" by Chris Wickham, see also "Medieval Europe", his brilliant introduction to the period in general
“Calamity and transition: Reimagining Italian trade in the Eleventh Century Mediterranean” by Romney Smith (published in Past and Present, Volume 228, 2015)