It seems that up until around the Enlightement/Industrial Revolution the successive Abbasid and Ottoman Empires tender to have an advantage over the Europeans both on the battlefield and in political courts. This has often created a sort of dissonance in my mind that is difficult to reconcile when the American classroom often presents western civilization as this long streak of unrivaled superiority to everything else, yet any look at actual history shows much of the european middles ages rife with major losses of territory and invasion by the Muslim world.
So what factored into this? Why did Europe take such a battering from the Ottomans and others during this time, and what were they lacking that their enemies had?
I'm going to speak of the Ottoman period.
First of all, it should be noted that this superiority only applied on land, at sea Christians first had parity and eventually naval superiority against Muslim Empires. This was especially true of "Blue-Water" navies in the Indian ocean. The Christians were of course victorious against the Ottomans at Lepanto in the Mediterranean, the Portuguese were able to smash the Ottoman navy in the Indian Ocean in the 1580s.
On land, the Turks had superiority until the 1640s because of
Whereas the Europeans suffered from
After 1640s however, the Europeans began to gain superiority on land through the development of effective field artillery, which was never matched by the Ottomans in kind. This eventually manifested itself in the defeat of the Ottomans by Russia and the Hapsburg, and the incursion of British into India.
Just one minor point: I wouldn't say that the Umayyads and the Abbasids necessarily had an extreme military advantage over the Europeans, as evidenced by their gains made in Europe. The only significant, long-term Western European conquest by the Caliphate was Southern Spain, and most of the reason that it fell so soon after it was invaded in AD 711 was because it had been ruled by a significantly fractured and declining Visigothic leadership that had barely endured attacks from both the Byzantines and various barbarian peoples in the previous centuries. The Visigothic state was never terribly strong and by the early 8th Century had undergone significant decentralization and depopulation. Sicily could be argued to be another loss, however, it was regained by the Normans after only a century in Muslim hands, and remained firmly controlled by European powers thereafter.
The Byzantines, while losing a great amount of territory in Egypt, North Africa, and Syria in the 7th Century, never lost their holdings in Europe or Asia Minor during the height of the Umayyad or Abbasid Caliphate, and they succeeded in established a relatively secure frontier by the 8th Century in the Taurus/Anti-Taurus ranges in Eastern Anatolia. While the Muslims sieged Constantinople several times, once in AD 674 and again in AD 717, they were not able to breach the Theodosian Walls, and were repulsed by the Byzantine Greek Fire, and therefore were never able to establish any significant holdings on the European continent for many centuries thereafter. Until the Ottoman capture of Gallipoli in AD 1354 following an earthquake which damaged that city's walls, no permanent holding in the Byzantine West (the European side of the Empire) had ever been under the control of the Muslims.
The Ottomans were basically the first Muslim state in about 700 years to conquer any new significant territory in Europe. And that, along with their unparalleled control of the Eastern Mediterranean and beyond, is what made them such a threat to the Western powers in the 16th and 17th Centuries.
The crusades and the reconquista were the major clashes between Europe and Islam during the middle ages, and taken all together it was more or less a draw. If you count the slow collapse of the Byzantine empire, then perhaps not, but I'm not sure how you feel that relates to "Western Europe".
The Turks gave a lot of trouble, but they were a steppe people and not originally Islamic. Eastern Europe was regularly clobbered by the steppe peoples--the Huns, the Magyars, the Mongols, the Turks, and others all gave a heap of trouble to established kingdoms and empires in the east, so it probably has more to do with horse peoples' warfare techniques than their religion. In fact, they beat up on Islam even more than they did Europe. The Seljuk Turks conquered Persia before they ever turned their attention westward (and later the Ottomans reconquered it again). The Ottomans also conquered Egypt, Arabia, and others. (And meanwhile, the Mongols arguably ended the Islamic Golden Age with the destruction of Baghdad.)
The height of Ottoman power did not precede, but coincided with the European Renaissance. The fall of Constantinople happened at roughly the same time as the Gutenberg Bible was being printed, and the (first) Siege of Vienna happened after Michelangelo's David and Sistine Chapel. So there may have been a rising tide that lifted both civilizations simultaneously. Ottoman military power did not really get pushed back until the late 17th, early 18th Centuries, by which time their power had been circumvented by the sea trade and they were simply not as important geopolitically as they once were.