If one looks at John Iliffe's Africans (2d ed., 2007) and other works, the low population density of much of the continent has a lot to do with it. Land tended to be more abundant than people to work it--thus why control over human beings was arguably more important than control over territory except in unusual circumstances like the Nile Valley or the Niger bend/inland delta--so the lure of heading over a horizon and hoping for land was simply not a factor. Where viable islands were visible from the mainland, they might well be colonized; think of Bioko here, for example, but not Sao Tome (imagine accent marks) or Principe. So look at the distance from the coast for these islands for your answer; I think the supposition you make is a good one. Beyond that, bear in mind also that African sailors likely may have visited some of these islands, but building communities there was not advantageous.

As for Madagascar, the same basic point holds until Austronesians are well established and the trading network fully elaborates south of Kilwa Kisiwani after 1000. Then you have the emergence of Swahili colonies on the north of the island, but that's in the context of the exchange and commodity production that characterizes the Swahili Coast.