Given the region he probaly wasn't white. Who was he modeled after? Why change it in the first place?
Jesus (and Mary as well) are often depicted as part of whatever culture is doing the iconography. They look African in Africa, Asian in Asia, Greek in Greece, Mexican in Mexico, etc.
Another spin off question, would Jesus have in actuality been brown since he was Jewish and Jews came from Egypt?