I know this is a bit of a broad question, so I might narrow it down a bit, but I have been wondering this a while. You have groups, like the Anatolian Greeks, the Bosniaks, the majority of the Coptics and Syrian Christians who ultimately abandoned Christianity for Islam (also the Zoroastrians, but that is broadening it up a bit again). However, there were cultures within those empires who did not convert. Like the Coats and the Serbs and the Bulgarians. What was different about these groups that kept them from being religiously assimilated?
This is a top level remark - so please let me know if I am breaking the rules.
While reading about early civilizations, I was struck by this phenomenon. While the early record is not very clear, I think three factors are important.
1 - Common sense. If your side is praying to its gods and the other to theirs. The winners will have a strong argument that theirs are the gods to be worshipped.
2 - Forced conversions. In many cases, the victors would build monuments to thank and worship their gods. They would expect the locals to participate in these occasions, regardless of belief. Islam allowed non-believers to exist, as long as they paid a tax. But earlier religions were not so organized.
3- Syncretism - the new gods would be merged with the old gods. A latter day example is Santeria or Santa Muerte which mixes native religious beliefs with Catholic beliefs. Again, this comes from unanswered prayers to the old saints and new cults promising miracles.