Just to be clear, I understand that my question is kinda stupid, and that the subject is HUGE. There are too many groups and peoples to give a simple answer. Still, I have wanted to start digging into this subject for years, and I just don't know where to start.
What I want to learn about, is really just everything, from the very broad to the specific: what ancient nations and kingdoms were there? What religions? Cultures? Languages? Tribes and etnicities? Anything and everything that I can possibly read, and more. Recommend me any and all books you think will be of interest in that regard.
I have no interest in ancient Egypt, though.
You should start off with the Book list section on Africa. As one of the flairs who wrote a lot of the synopses for it, I think it is a pretty good resource.
I know that /u/Khosikulu likes Iliffe's book Africans, the History of a Continent. I have recently bought Ehret's book and find it quite good.
To be honest, you can skip the UNESCO books. They are quite old by now, and you would be better served by the more recent work.
And of course, going beyond the "General" section, once you get into the more specific regional or topical subheadings, lots of books deal with pre-colonial topics.