I can't give an answer for the lack of adoption of Arabic in every single case, but I can give some context as to why and how this happened it some cases as context for wider trends.

Islam largely spread by conquest in its early history. A very thin layer of conquerers from the Arab peninsula ruled over an existing population. Conquest of this kind has often led to the adoption of the language of the ruling class - that is why we have Latinate languages from Spain to Italy and why English people speak a Germanic language imported by the Angles and Saxons rather the Celtic language they originally had before those conquests.

During Islamic conquests the local population faced a choice if they weren't People of the Book - conversion or death. Most people chose to convert. Since Islam requires more familiarity with Arabic than other religions do of their original language, this may have even further increased the rate of language adoption by the local population compared to a "usual" conquest.

So what would retard this natural process in some places and not others? One notable example, as you mentioned, is Persia. Why did Persia retain native languages? One component is probably among the largest proportion of "People of the Book" of any conquered country - Zoroastrians were not required to convert and could continue to practice, leaving a protected pool of native languages. Persia also had a relatively robust, cohesive cultural and political system in the Sasanian empire in their own homelands when compared to North African conquests.

Another example is Turkey - an Islamic country that does not speak Arabic for a completely different set of reasons. Turkey was conquered by a group of Turkic people that originated in the Eurasian steppe. They were not present during the initial Muslim conquests, and converted to Islam after later exposure to it -but this time as conquerers themselves. This means they did not get pressure to adopt the Arabic language, but rather exerted pressure to adopt Turkic languages.