It's mainly about climate and population density. Although it is possible there were ancient cities in Africa whose ruins have simply been lost.

1. Look at that map you linked to, for the first few thousand years, all the cities are along the same line of latitude. Northern Europe, Russia, and southeast Asia are also relatively 'late' to develop cities.

Cities first developed due to the advent of farming. Farming was first developed in Mesopotamia, and from there it spread. However, farming techniques must be adapted to the climate and crops available. Africa had a much different climate and different crops, which required a lot of expertise to cultivate due to Africa's harsh climate. This meant that the 'agricultural revolution' was slower to take hold in Africa, thus less cities. The earliest cities were all in the Mediterranean climate, whereas sub-Saharan Africa was more forests or savanna, so agriculture had to be adapted, which took a while.

Even then agriculture was limited. Things like coffee and bananas were farmed in Ethiopia, and in western Africa they grew yams and rice.

2. Africa had a lower population density throughout history. Africa is huge and the population is spread out. Cities developed through agriculture and the need to defend against other groups. The more spread out groups are, the less likely they are to clash, and it becomes less likely they will build cities. If you look at the areas where cities developed, they are largely fertile river valleys surrounded by desert, so there was more incentive to fight over the best land.(otherwise you'd be stuck in the desert and unable to farm)

Source: "Africa in World History" by Erik Gilbert and Jonathan T. Reynolds.