Because of a little something called the Sahara Desert. The only real routes of expansion into Africa were along the southern Moroccan coast (which was desolate, sparsely inhabited all the way up to the late 18th century, when the Moroccan government began to encourage irrigation projects there) and via the Nile Valley. However, the rough terrain and relative poverty of the Sudanese and Ethiopian highlands meant that it was not really worth challenging the Kushite and Axumite lords of the upper Nile region for control. Then a far more economical policy was to maintain a web of buffer vassal states to hold off the more powerful kings in the southeast.

In other words, both the paths into Africa were impractical to use for any sizeable party in ancient times, much less a host of men, animals, and camp followers the size and scope of several Roman legions. The regions were, for lack of a better term, nigh on unconquerable in ancient conditions. Even a thousand years later, the Ottoman Empire found it extremely difficult to extend its rule south of the Wadi Halfa, and by then the introduction of the camel had led to the establishment of proper roads due to the trade explosion.

Though the lack of true oceangoing ships prevented any Roman colonisation of the West Coast of Africa, in theory it would have been possible for them to preempt the Arab colonisation of Eastern Africa even with galleys, establishing tradeposts around the Horn of Africa. However, the only really suitable port, Baranis on the Red Sea (Berenice Troglodytica)... Simply put, it did not have any trees, at least not of sufficient size and tensile strength to build proper vessels. It functioned as an emporium for the Red Sea trade, but could not construct vessels of its own. In other words, for a Roman living in Alexandria to set up a trade post e.g. near Zanzibar, he would first have to travel down the Nile with hundreds of settlers, cross the desert mountains filled with Tuaregs, Berbers, and probably bandits, to get to Baranis, and there buy enough ships to take his entire party and supplies on a practically blind four-month journey into uncharted waters. Once there, he would have to find a trade goods worth exporting all the way back to Baranis, then set up and fortify his colony while establishing farmland and regular trade routes with Baranis, all the while probably fending off hostile natives... It was simply not worth it, when spices and ivory could be obtained far more cheaply from Indian, Arab and Ethiopian merchants.

If you want a more detailed answer, I can recommend some books and articles.