I was always under the impression that the Muslims were iconoclastic.
They didn't leave them alone, many of the mosaics within the Hagia Sophia were covered over with plaster, though not destroyed. Restorations in the 1800s and 1900s uncovered some of them.