Sorry in advance if this sounds offensive / ignorant. I am asking because his journey took place well before the Atlantic Slave Trade started. So I am not sure what the sub-Saharan Africans' grievance would have been against "white people," or what their concept of "white people" even was. I guess "white" meant Arab to them? But then, why wouldn't he just say, "contempt for Arabs"? Did Europeans even go to those lands in that time? Alternatively: is this contempt noted by any other travelers of that time, or was ibn Battuta just reading too much into his encounter? Or perhaps a mistranslation?
In that era, everyone subjugated everyone (said with vast generalizations). But the fact remains, for example, that African kingdoms bought and sold African people themselves.
The racial component of slavery before the era of imperial Europe was highly dependent on the culture or even the specific individual. For example, the Arab slave trade was not always motivated by racist attitudes (dependent on individuals). The Arabs generally seem to have been equal opportunity subjugators.
Thus in answering your question, it's impossible to understand why this particular governor, an individual living in a particular culture, had contempt for white people. It could literally be anything. Is Ibn Battuta assuming that the whole culture is racist as a result of this one person? Are there any other parts of Ibn Battuta's journey that suggest racist attitudes? It could even be something as simple as the governor trying to show off his power and superiority of office.
There are a LOT of books out there on the history of slavery, but one that I've referred to for a very basic introduction is:
Sylvester, T. (1999). Slavery throughout the History - Almanac. 3 Volumes. Gale Group: London.