Because...they don't?

Sorry, I have to reject the premise of your question. There are multiple Amazigh communities that stretch from the Mediterranean sea south to Niger and Mali. A Kabyle person from Algeria like Zinedine Zidane has different facial features and skin tone from a tuareg from Niger. However, both Kabyle and Tuareg peoples consider themselves Amazigh (or Berber).

So, "Berbers" have a variety of appearances, and so do other African peoples, and it is not accurate to say "Berbers look different from other Africans"