Did they ever attempt to invade? No. There is no evidence for an invasion of sub-saharan Africa by the forces of the caliphate. What is known is that islamic missionaries would often travel with merchant caravans traveling to the salt mines of Mali and the Sahel. It is for this reason we have a sizable muslim majority in West Africa which lead to the most famous King of Mali, Mansa Musa, traveling in pilgrimage to mecca in the 14th century.
Trade along the east African coast also led to significant Islamic influence as areas such as modern day Mozambique, Kenya, Somali, and Madigascar now have significant Islamic populations dating back to the 9th century CE.