Arabic is often referred to as a "diglossic" language in that it has both a "high" register of classical or modern standard as well as a "low" register. This is relatively common as these things go for any language that maintains some kind of literary cannon or "accepted" mode of speech over several centuries like Greek or Latin.

But that's not really the whole story as Arabic maintains not only a "High" register and single "Low" register but actually has multiple regional "low" registers, as you note, including Egyptian Arabic and Levantine Arabic.

Moreover, within those regions, you will also have significant variation. /u/The_Turk2 uses the examples of peasants and the urban elite, which is part of it, but there's also significant elements of code switching. A medical student in Cairo will almost certainly sound indistinguishable in casual speech from any of their less educated friends in a social setting, but would also be capable of giving a presentation to a medical board in "formal" Arabic (or, for that matter, in English.) Likewise, a comedy presenter like Bassem Yousef would present his show in a more informal register than the newsreaders on Radio Misr.

Where I disagree with /u/The_Turk2 is that this is not a question of accents. Formal Egyptian Arabic will differ from Formal Levantine Arabic largely as a matter of accent if you're watching a news program, so in that sense, yes, formal communication is relatively straight forward. But even well educated speakers in casual conversation will differ markedly not just in pronunciation but in vocabulary and grammar as well, the point of unintelligibility between informal registers only a few countries over unless both speakers resort to a higher, standardized register.

As to your question for why this is the case there are a few reasons. One is that maintenance of the Quran as the absolute standard of linguistic perfection across the centuries more or less guaranteed that a higher and lower register would develop over time, as it has in other languages that maintain a classical canon.

But there is also a political aspect to this as well, because even though Moroccan Arabic is, for all intents and purposes, a different language from Iraqi Arabic, the ideology of pan-Arabism is to continue to insist that these are merely differences of accent.

As I've said, this works to an extent for formal registers, and as a result there's been a proliferation of "pan-Arab" TV networks and films, particularly in recent decades as the audience for such content has expanded. That's why Al-Jazeera and Al-Arabiya broadcast in MSA and it's why Disney has moved to create its films in MSA after decades of dubbing them in Egyptian dialect.

Sources: Not to break the 20 year rule but just as an example of how this process works and some of the differences between MSA and informal see the following New Yorker article, which has the benefit of also being amusing: http://www.newyorker.com/books/page-turner/translating-frozen-into-arabic

Much has been written on Arabic diglossia and polyglossia, I'm having trouble finding the particular article that I'm looking for but as a holder see the following: https://books.google.co.uk/books?id=ekzP5NrGkNYC&pg=PA69&lpg=PA69&dq=multiglossia&source=bl&ots=84UvfgIEk7&sig=GS_9K5oGVs-aa2CZLcgdathtBZI&hl=en&sa=X&ved=0CCcQ6AEwAmoVChMI8r6b-KqPyQIVgW8UCh1crQlc#v=onepage&q=multiglossia&f=false