There is a large genetic continuity between modern North African populations and their pre-Islamic ancestors. Studies of Egyptian DNA show that waves of invaders and settlers have accounted for only about 10 percent of the population's DNA. As in many other modern Arab nations, the vast majority of people are descended from natives who adopted Islam and the Arabic language. I will defer to a specialist on any questions about that process.
Although the Berbers of the Maghreb, since Numidian times, have seen invasions and colonization by Vandals, "Byzantines," and Arabs, several of the ethnic Berber groups show remarkable similarity with DNA found in fossils. The "Berber gene" is known as E-M81, orE1b1b1b.
"North Africa" is, of course, a broad term. Moroccan Berbers are noticeably distinct from Tuaregs in Northern Mali, who look, for lack of a better term, half-black and half-Arab. Other groups, like the Zenaga, although a deal south from the coast, have a fairly generic olive/Mediterranean look.
Naturally, there was migration, particularly between Kush (modern day Sudan) and Egypt and less so across the Sahara, and there would have been blacks and admixture between populations. The Nubian Pharaohs of the Twenty Fifty Dynasty of Egypt in 8th-7th century B.C.E. would be the closest thing to black rulers. If the question is whether North Africans as a population looked like Sub-Saharan Africans and were bred out of existence to look generically Arabic, I'd have to say no, not really. A mix of predominantly Berbers and then German/Spanish Vandals and Byzantine Greeks were probably more likely to describe the population than what people see as "black," Sub-Saharan groups like the Khoi Khoi or Ndebele.
Some of the more recent books to look into are:
Alessandro Achilli et al, Saami and Berbers—An Unexpected Mitochondrial DNA Link, American Society of Human Genetics, v.76(5), May 2005,
J. Desanges, "The proto-Berbers" 236-245, at 237, in General History of Africa, v.II Ancient Civilizations of Africa (UNESCO 1990).
Arredi, Barbara; Poloni, Estella S.; Paracchini, Silvia; Zerjal, Tatiana; Dahmani, M. Fathallah; Makrelouf, Mohamed; Vincenzo, L. Pascali; Novelletto, Andrea; Tyler-Smith, Chris (June 7, 2004). "A Predominantly Neolithic Origin for Y-Chromosomal DNA Variation in North Africa". Am. J. Hum.