Honestly? It's incredibly difficult to tell. I researched this topic before, and the lack of religious demographic information makes it extremely difficult. So I can't give you a solid answer, but I can give you some points to consider.

While the population of Christian Europe was definitely less than that of the far-flung Muslim states, it's important to remember how multi-religious the Muslim states were in comparison to Christian Europe. The Mughal Empire, while Muslim, ruled over huge numbers of Hindus (and a small number of Christians in Kerala). The Middle East had large numbers of Christians (and also Jews and Zorastrians). Less and less over time, but even today, 1300 years after its conquest by Muslims, Egypt is still 10% Christian. In the Middle Ages, there were a lot more Christians in the Middle East than there are now. Muslim rulers were generally happy to rule over a multi-religious, multi-cultural land, where Muslims were the elite, but not necessarily everyone, or even always the majority. I'm sure you've seen Iraqi Christians mentioned on the news? Most of those are members of the Assyrian Church or the Syriac Church or the Chaldean Church... there are a lot of Middle Eastern denominations.

There was a significant Nestorian Christian minority in Persia, and they sent missionaries out in all directions, even after Persia's conquest by the Arabs. While Nestorians never became the majority anywhere, they were far-flung enough that their total number was still significant. For example, four Mongol Khans were the sons of Sorghatani Beki, an extremely influential Christian Mongol princess, and her husband Tolui (son of Genghis Khan). When the Mongol Ilkhanate wanted to negotiate with the Europeans for an alliance against the Mamluks of Egypt, they sent a Uigher (from what is now Xinjiang in western China) Christian Bishop.

Ethiopia was thoroughly Christianised a long time before Islam, and continued to be all through the Middle Ages.

Essentially, it is quite possible that there were more Muslims than Christians in the Middle Ages; I can't really give you a solid answer. I consider it likely at some point, as the Middle East and Central Asia slowly became more Muslim. The Muslim-held parts of the world were, after all, much bigger in population than Europe. But it's worth remembering that a)a lot of people who lived in Muslim-held areas were not Muslim, and b) Christianity was a far-flung religion, it simply had the most power in Europe. There are denominations today in India, Iran, Iraq, Syria, Lebanon, Ethiopia, Armenia, etc, who trace their history back through to early Christianity with no links to Europe at all.