To my knowledge, the Jews were pushed from Palestine in the first century CE, under Roman occupation. What happened when Rome began to collapse a few hundred years later? Did Palestine remain firmly under foreign control until the twentieth century?
Again, to my knowledge, the first sincere effort at returning to the Levant began in the 19th century. Why did it take so long? Since Rome was the power that initiated the diaspora, I might imagine that Rome's collapse (or its loss of foreign territory) could have spurred a migration back to the Levant.
Were other powers occupying portions of Palestine during the middle ages not amenable to the Jews' return (e.g., Byzantines, Persians, Arabs)?
As a related question, didn't the Jews displaced by the first diaspora (in Babylon) return to Israel after the conquest of Cyrus the Great? What makes the first diaspora (and its return) different from the second?
It's actually a rather commonly held but false belief that the Jews as a people were exiled from Palestine under the Roman Empire. What the Romans did do was destroy Jewish political independence and identity, and we know that Jews were banned from the city of Jerusalem, many towns were destroyed, and many were sold as slaves or executed in the rebellions. However there's no evidence of a true 'exile', it's more a metaphor for political subjugation.
So as we have no evidence of a large scale exile, the best explanation as to why Jews became a minority in the land of Palestine is the same one as why Christians did after the Muslim conquest - conversion to other religions (Christianity and Islam), emigration, and in-migration of other peoples.
This explanation above is mostly from Schlomo Sand's 'The Invention of the Jewish People'. He's politically far on the 'anti-nationalist' side and I think he's mistaken on some other things he says, but as far as I'm aware the historical consensus is indeed that there was never a true deportation of Jews as a people from Palestine during the Roman period. Some support for this comes from this review of the book by Simon Schama:
"But it is a long time since any serious historian argued that following the destruction of the Second Temple, the Romans emptied Judea. But what the Romans did do, following the Jewish revolt of AD66-70 and even more exhaustively after a second rebellion in AD135, was every bit as traumatic: an act of cultural and social annihilation – mass slaughter and widespread enslavement. But there was also the mass extirpation of everything that constituted Jewish religion and culture; the renaming of Jerusalem as Aelia Capitolina, the obliteration of the Temple, the prohibition on rituals and prayers. Sand asserts, correctly, that an unknowable number of Jews remained in what the Romans called Palestina."
The reason why Jews from Europe 'returned' to the Levant from the 19th century is really because of the rise of the idea of nationalism in Europe. This provided both pull factors (a nation-state for each ethnic group was now seen as desirable) and push factors (ethnically defined nation-states often seemed to have no place for the Jews, though this was not the case so much in liberal states like Britain and France).