I'll try to answer your question, sorry if it doesn't satisfy you

First of all, devşirme recruits are rarely made into eunuchs, most of them are purposefully trained to become soldiers or court officials. There are male "white" eunuchs in the Ottoman Palace, but these eunuchs are acquired from prisoners of war or the Black Sea/Caucasian slave trade. We'll talk about them and the "African" eunuchs in a bit

To start, devşirme and the harem are two different institutions. The former is to create a loyal military (later palace staff) personnel, while the harem eunuchs are to guard harem (closed) sanctions of palaces or holy sites. There are both practical and "racial" reason on why East Africans are chosen to be harem eunuchs.

The Ottoman Empire inherited the eunuch system from the Mamluk Sultanate, with some influences from the Byzantines. Under the Mamluks, eunuchs of East African, Balkan, and Indian origins served as guardians to the women's quarters and as teachers/guardians at the barracks, a practice they inherited from their Fatimid and Abbasid predecessors. While Ottoman eunuchs didn't linger in the barracks, white eunuchs did supervise the Ottoman corps of palace pages, while African eunuchs were in charge of the princes’ early education. Just like their predecessors, the Ottomans obtained their eunuch from their peripheral regions, and East Africa is naturally one of them. East Africa was adjacent to Ottoman territory after the Ottomans’ 1517 conquest of Egypt, and sometimes the Ottomans themselves ruled parts of it. This, added with the preexisting slave trade networks made East Africa the perfect choice for source of eunuchs for the Ottoman Empire.

Besides those practical reasons, there are also reasons grounded more in indigenous racism. Early Ottoman elites, inheriting traditions from the Abbasids saw "black" East Africans as "backwards" or "depraved" caused by the "Curse of Ham". At the same time, East African "races" were regarded as having their own specific positive qualities. Ethiopian males for example, were regarded intelligent and trustworthy, suitable for household tasks and record-keeping which is why they become the preferred pool for Palace eunuchs. If you want more detail on this just tell me.

Now on to the "white" eunuchs. The Topkapi Palace can be broadly divided into sections: The outer/first court which was accessible to the public, the second court where the palace troops and officials reside, and the third court which was restricted only to the Sultan and his aides. In the 16th century, the harem was separated from the third court. The African and White eunuchs were associated with the harem and third court respectively, specifically African eunuchs guarded Darüssaade, corridor entrance to the harem while white eunuchs guarded the Babüssaade, threshold in front of the Sultan's private chambers. On the reason of this divide, it's because white eunuchs had earlier roots as companion or guards for the Sultan. During the reign of the second Ottoman Sultan Orhan, Byzantine slaves/eunuchs served as companions to the Sultan while, at the same time, controlling access to him. This was more or less "codified" by Murad II who stationed white eunuchs to guard the Palace entrance. This didn't change with the entrance of African eunuchs, who guarded the harem.

Sources:

The Chief Eunuch of The Ottoman Harem by Jane Hathaway

The Making of Ottoman Court Eunuchs by A. Ezgi Dikici

Celibate and Childless Men in Power edited Almut Höfert, Matthew M. Mesley, and Serena Tolino