A fair amount of lanugages whose speakers are predominantly Muslim (like Circassian, Turkish, Crimean Tatar, Tajiki, Malay, and others) used Arabic in the past as their writing system. Nowadays, most of these languages (with the exception of Persian, some dialects of Kurdish, and Malay in some places) mostly use Latin or Cyrillic. Was there any reason having to do with linguistics (with it being cumbersome to use) or political reasons for this fairly large shift?
I can’t give a general answer to this I only know the Turkish reasons. Maybe some of the reasons are valid in other Turkish languages and countries.
Ottomans were using Arabic script with Persian modifications since Arabic doesn’t have ç, j and p they were borrowed from Persians but still ğ was missing because Persians don’t have ğ either. The main problem is that you can’t really write Turkish with Arabic script because Turkish is based on vowels and suffixes but Arabic is based on consonants. You can easily make a mistake or cause a misunderstanding.
Ottoman official language is Turkish but we call it Ottoman Turkish or just Ottoman because it is another form of Turkish with so many Arabian and Persian loanwords and grammar rules. Ottoman court and bureaucracy was made up different people from different cultural background. For example a crown prince could learn Turkish/Serbian/Greek/Russian/Bulgarian/Circassian or any other language from his mother then he would speak Turkish or Ottoman Turkish with father then he would learn Arabic or Persian as a foreign language as a kid. This was an advantage in first 200 years. Some polyglots appeared as Sultan and there were benefits of it but as the country grew many different languages entered the daily life of crown princes and bureaucracy. So this language called Ottoman Turkish appeared. During the 16th century Ottomans conquered many different lands and inherited the words being spoken. Ottoman language has been a composite language after that time and an ordinary Turkish person wouldn’t even understand it completely. At the 19th century Ottoman Turkish has been a broken language that is very hard to learn and understand. Some of the magazines had started using Arabic and French instead and Ottomans were really isolated from their own citizens because of a big language barrier. Also they didn’t care much about their people either. Most of the country was undereducated and had extreme poverty. Literacy rate was very low. Having islamic schools didn’t help either because Quran had no translation and people were learning Arabic alphabet mostly.
Since the language got broken there were also language reform ideas. Some people argued that Ottomans should swap to Latin alphabet but couldn’t make a real alphabet after that they started using French script but it ended the same way and became broken like Arabic script Turkish. Some people argued it is possible to reform Ottoman alphabet but that never happened either.
After the war of independence we declared republic and started republican reforms. We also came up with a completely new alphabet based on Latin script in 1928 and we have founded The Turkish Language Association (Türk Dil Kurumu). After that literacy rate skyrocketed and Association has removed most of the loanwords. There is no language problem since then.
u/kcapoorv asked Why are Bahasa Indonesia and Malay written in Latin script? My answer explains how the Latin script eventually supplanted the Arabic-based Jawi script for Malay in Peninsula Malaysia. I can't answer for the other languages, unfortunately.