When discussing the rise of political Islam, we really have to go back to the decline of the Ottoman Empire. People in the Middle East were not oblivious to this happening, and they certainly noticed that Western Powers were beginning to consistently overpower the Islamic world. Naturally, this created quite the ideological debate as to what should be done, resulting in the development of Ottomanism along with early variants of pan-Arabism and pan-Islamism. Note that, at this time (late 19th- early 20th century), these discussions are happening overwhelmingly among elites. Most people are still simply living with the reality of the Ottoman state and not challenging the status quo.

“Ottomanism” attempted to reform the Ottoman Empire into a state that could compete on equal footing with the Western “Great Powers”, and develop an Ottoman national identity based in civic (that is, not ethnic or religious) nationalism. This obviously became obsolete with the defeat and subsequent collapse of the Ottoman Empire in WWI (and in any case, Ottoman leadership, under the Three Pashas, had been moving away from it in favor of Turkish chauvinism).

After WWI, the Great Powers partitioned the Middle East into colonial mandates (such as French Syria and British Palestine) and officially independent, but heavily influenced regional states (such as Iraq and Egypt, both under heavy British influence). During this time, resentment towards the West grows further, while Arab nationalism grows. This nationalism takes two main forms; one favoring a nation based on al-watan (“the homeland”, ie the creation of multiple independent states based on local nationalisms), and another favoring al-ummah (“the nation/people”, working towards a single pan-Arab state).

Fast forward to the aftermath of WWII, and many of the former colonies become independent, while the European grip on pre-existing states (like, again, Egypt and Iraq) begins to slacken. However, Israeli independence (and even before that, mass Jewish immigration to Mandatory Palestine) further enraged many Arabs and was seen as another insult by the West. At first, many of these states are more heavily influenced by local al-watan nationalism than pan-Arab al-ummah nationalism. After all, the established local elites don’t want to lose their power, and many take more pride in their own country’s history than in the concept of a united arab nation.

Both local nationalism and Arab pan-nationalism, share an important quality in that they’re largely secular. This, along with continued feelings that the “Christian” West is trampling over the region, leads to backlash and political mobilization by conservative Muslims. Led by figures like Hassan al-Banna, this creates the nascent Islamist movement. They’re not very powerful yet, with nationalism still being much more prominent, but keep this in mind.

The humiliation of the Arab defeat in the first Arab-Israeli War, along with the continued alliance of many Arab governments to the West, led to spiraling unrest. Arab pan-nationalists, favorable to the Soviet Union, seized power in many countries, starting in Egypt in 1952. Thus began the Arab Cold War: a conflict between the generally pro-Soviet, pan-nationalist, and nominally republican Arab states and their generally pro-American, local nationalist, and monarchist rivals. The sides aren’t totally set in stone (for instance, the monarchists will sometimes tip the hat towards pan-nationalism), but it’s an important frame for what’s going on in the Middle East during the mid-20th century.

The pan-nationalists are euphoric that they’re on the right side of history and are going to win. They believe they will liberate all Arab countries from traditionalism, drive out imperialist influence, destroy Israel, and create a pan-Arab superpower. This is further reinforced by “victories” like the Bizrete and Suez Crises.

Then comes 1967. After years of anticipation, the climatic clash with the hated State of Israel occurs, with the pan-nationalist Egypt leading the charge.

And the Arabs lose. Very, very badly.

This is very traumatic for a lot of people in the Arab world. They’ve been trounced by their “enemies” again. Not only that, but many Arab economies are suffering stagnation, and attempts at creating unified Arab states always fall through. Gamal Abdel-Nasser, the charismatic dictator of Egypt and figurehead of Arab pan-nationalism, also dies in 1970. Arab nationalism, in both its local and pan-nationalist forms, seems to be a failure. This creates a search for alternatives, and Islamism, which had already been developing as a minority ideology for decades, was a ready alternative.

The 1970s see a variety of other factors come into play. Saudi Arabia began supplying more funding to conservative religious movements at this time, and while a notable factor, I don’t think it’s the central cause of the rise of Islamism. There’s a more broad turn towards religion in general, and many of the Islamist movements that develop aren’t associated with Saudi Wahhabism. Some aren’t even Sunni.

That brings us to 1979 and two titanic events: the Islamic Revolution in Iran and the Soviet invasion of Afghanistan. The Islamic revolutionaries in Tehran both openly and covertly tried to export their ideology abroad, mainly to other Shiite communities. This obviously directly fueled Shiite Islamism, but also fueled Sunni Islamism as a reaction.

The Soviet invasion, meanwhile, was an attack by an openly atheist, anti-religious super-power against a small Muslim country, and an exceedingly brutal attack at that (civilian casualties in the Soviet War were much, much higher than under the later American occupation, which was itself heavily criticized). This strengthened the self-identification of many Muslims with a global Islamic community. It also led to the “Arab Afghan” movement, where mostly Arab Muslims traveled to Afghanistan to join the mujahideen in their fight against the Soviets. One those young fanatics was a man from a wealthy Saudi Arabian family, named Osama Bin Laden.

The “Arab Afghan” jihadis were generally inconsequential in the war with the Soviets. They were seen as ineffective by the United States and Pakistan, which supported local Afghan rebels instead. Contrary to the popular urban legend, there is no evidence the CIA ever worked with Osama Bin Laden, let alone “created” al-Qaeda.

More important than the direct participation of Arab Islamists in the war was the impact it had on their organization and ideology. Many Islamists formed tran-national connections in Afghanistan that helped form future terror networks. The war also led to a further radicalization and militarization of many Islamist ideologies, and the apparent “victory” of Islam in the war further fueled Islamism’s popularity.

Now we’re already in the time frame where many of the terrorist groups you mention have begun operating, so I’ll stop there. Needless to say, this is a very complicated topic (the rise of ideological movements always is), but I hope this helps you understand it a bit better.