Pragmatism, racism, politics, and religion. By the founding of the Abbasid Caliphate, Eurasia was deeply entrenched in what scholars call the "millennium of the mounted archer", a period where horse archery was the most effective martial art throughout most of the continent. Many countries we don't often associate with horse archery made it a cornerstone of their doctrine: Byzantine historian Procopius, living in the age of Justinian, said of horse archers that "all our victories were a result of this very arm". In the highly maneuverable environment of mounted combat, range was everything. The initial advantage of the lance on horseback was the ability to engage enemy infantry and cavalry at greater range. And, of course, no horseman had greater range than the horse archer.
While they proved fierce cavalrymen, Arabs were not traditionally mounted archers. In the same way the Byzantines hired Avar auxiliaries to master this new military doctrine, the Arabs needed foreign recruits as well. The Turks were the natural candidates, both because they were the recipients of positive racism (in the words of Timothy May, the Arabs saw them as "more natural warriors" than the Persians or themselves) and because they were predominantly pagan, and Muslims could not be enslaved. In the early Abbasid period, most Turks still worshipped Tengri and other traditional deities. Even after their conversion to Islam, slave traders could still claim their captives were Jews, Buddhists, Christians, Pagans, or "improper Muslims". The latter accusation was easy to make, since the steppe nomads in this period were almost universally illiterate and had no real comprehension of the texts or doctrines of the religions they practiced. They retained pagan traditions in their practice of Islam as a result. To this day Kazakhstan, for example, still has pagan witch doctors called Baksy.
Turks were also not prohibitively expensive because they were ready to participate in the system. While some Mamluks were adult males captured by tribal enemies, most were boys as young as 7. Many of these were sold by their own fathers: the nomads were prolific and were almost always running an unsustainable population surplus. This is a surprising fact in a society that revolved around the concepts of pride and honor, and indicates that the nomads did not see Mamluk service as akin to other kinds of slavery. Contra popular belief, most pre-Ottoman slave soldiers did not remain bonded for life. They obtained their freedom (at least in practice if not officially) after completing their military training. After this point, they had far more opportunity to acquire riches and power than what the steppe could offer.
Moreover, the distinction between "free" and "bonded" labor mattered very little to steppe people. Every man was a herder, and he was also a warrior. All his property he shared with a group of three to five families he traveled with and made winter quarters with. If his tribe went to war, he was expected to fight. If he was captured, he would most likely be convinced to join his enemy. He would then marry into their kinship network and start a new family. Societal regulations on the steppe were regulated by the threat of violence, not laws and rights. In other words, wherever a man went, politics followed him, and no one thought of himself as truly "free". When mamluks arrived in the Middle East, they quickly adjusted in the same way they would have adjusted had they been captured by tribal enemies: by establishing their own patronage networks and playing local politics.
The decision to buy boys young was motivated by cost, politics, and training. Boys were cheaper than trained warriors, and on the steppe learned to ride before they could walk: the key skill they needed to become cavalrymen was already there. At that point, the traditional steppe training of a warrior would no longer do. Steppe armies were masses of poorly armored men who herded part time and fought part time. Their training consisted of hunts, which, while useful in teaching marksmanship and even military tactics (the Mongol art of war, for example, was heavily influenced by mass hunting techniques like the nerge), it meant their focus was divided. The Muslims wanted the greatest value for their dirham. They wanted their Mamluks to live, breathe, and sleep war. They armored them, equipped them with lances as well as bows, and trained them to fight with all kinds of weapons. Finally, Mamluks bought young were seen as more politically reliable. A grown man had tribal connections, and was a creature of the steppe. Life there was violent and unpredictable, but it was what he was used to. He could get homesick. A boy, in contrast, was "adopted" by his Mamluk unit: they became his tribe and the Caliphate was his home.
Like most groups of imported warriors, the Mamluks were initially divorced from local politics. At first, this was one of their main selling points. Abbasid noblemen and the Caliph himself used them to "outflank" traditional enemies in the power elite and sever unreliable relationships. By the end of the Anarchy at Samarra (861-70), however, the Mamluks had acquired a well deserved reputation for being just as treacherous as their masters. Still, rulers continued to turn to Mamluks because they had no one else. It wasn't that the Abbasid Caliphs, the Persian dynasties that usurped their authority, the Fatimids, and so on perceived no threat from their Mamluks. Rather, the short-term goal was to undermine political enemies and they or their descendants would deal with the Mamluk threat later.
In the 14th century and beyond, the pattern of slave import in the northern Muslim world greatly changed. While Egypt - by that time ruled by Mamluks - continued to import nomads to shore up its cavalry arm, the Ottomans and Iranians primarily enslaved sedentary Christians for military service. This was both because of politics and because of supply and demand. After 1071, Iran and Anatolia were both overrun by nomadic Turks. This only increased after the rise of the Mongol Empire, which drove dozens of Turkmen tribes (including the Ottomans themselves) Westwards. What the Ottomans lacked in the 14th century - something Mesut Uyar's overview of their military history goes into great detail about - was reliable, disciplined infantry with which to conduct sieges.
The ghilman system was unique to the Muslim world because other models permeated elsewhere. During the millennium of the mounted archer, all powers adjacent to the steppe, from Central Europe to China, were trying to learn this way of war, but each had their own method. Europeans tended to follow the Roman foederati model, where they would grant lands and political power to nomads in exchange for military service. When the People's Crusade reached Byzantine Serbia en route to Constantinople, they were shocked to find that the area was being governed by a Turk. After being ravaged by the Mongols, Hungary adopted an even more radical approach: they welcomed the vanquished Khan of the Cumans, married his family into the Hungarian royal family, and granted him the Pannonian pastures as a fief.