There are two things to bear in mind when explaining this discrepancy.

The first is that Africa south of the Sahara is not ecologically uniform. That may be obvious to just about everyone (including you) but it's easy to forget in relation to the way disease vectors work. Roughly everywhere north of the Tropic of Capricorn and south of the Sahara, the diseases that plagued European visitors (and even people of African descent who lacked prior exposure) were present, but their strength wasn't uniform. Those Europeans who arrived at the coast appeared in the forest belt or the wooded savanna, else the well-watered monsoon coasts in the east, where parasite loads for the killer diseases were quite high. So not only were Europeans naive to African parasitic diseases, but their exposure was sudden and massive in most areas. Staying on the coast away from the bulk of the mainland was the safest thing, though it still had peril; islands were even better if a trading power could colonize them.

In these areas, even African populations were relatively sparse on the whole, and usually in areas that had been cleared or were situated at a slightly higher elevation. Diets and some biological factors mitigated their risk, but the mortality rate for small children remained enormous. If you survived childhood, however, you could live quite a long life. But this meant that African traders (and later Eurafricans, people of mixed descent who had acquired immunity) had control of the interior trade routes. Those few Europeans who acquired immunity still tended to be cautious, and were rare enough away from the trade castles and posts that they ran significant risk of other more human dangers. That bit is part of the reason that Rene Caillie (imagine the diacriticals) had to carefully acclimate himself to the ecology as well as the culture for years before making his successful trip to Timbuktu in the 1820s, which made him the first European known to arrive there.

The second factor is that, generally, Arabs weren't doing the bulk of trading and spreading of Islam throughout West Africa. Berber caravans handled most of the interaction, although Arabians (and other Muslims from Asia) did visit the cities and states at the southern edge of the Sahara. Generally these visitors stayed at the major trade depots and centers of Islamic scholarship at the northern bend of the Niger, Lake Chad, or along the Nile, where the combination of carrying capacity on the land and climate conditions made it favorable for building and supporting cities (Jene, Timbuktu, Gao, etc). A few went further, but on the whole, trading castes (like the Wangara and Dyula) were important and drawn from more local populations of Muslims. Islam also didn't spread to the broad countryside south of the Sahara until after the 16th century in the western reaches, but remained an urban and elite sort of creed; when you got too close to the forest belt, Islam made very weak inroads even then. (The Horn is a different story I won't get into here, because of its proximity to Arabia and dry conditions.) Still, the peril was less in any case, because these travelers across the Sahara were crossing into areas where the disease load was far lighter. It's also very likely that some had immunity, acquired from Asia, to a few diseases such as malaria, but the chronicles I've read don't really talk about that and we don't have numbers the way we do for European deaths.

So it's a combination of where they were entering, how far groups of people actually went into the hinterland, and where the exchange of goods and ideas took place relative to the hazardous disease vectors' ranges.