Pre-Islamic Arabia and even early Islamic Arabia did not have many grand or elaborate temples and later mosques (the kabba wasn't that elaborate). So when did they start building some of their more glamorous structures?
Interesting question! The oldest mosque with elaborate decoration that survives in a reasonably intact fashion is the Great Mosque of Damascus from about 706, although the Dome of The Rock was built c. 691 and it is possible and maybe probable that similarly elaborate mosques were built around the same time as this and have since been lost or wholly rebuilt-for instance, very little of the original decoration of the late 7th century Great Mosque of Kufa survives. It does seem however that the earliest elaborate mosques in the West date to Umayyad times and in particular Syria. Now why Umayyad Syria? Part of this is simply patronage; the Umayyads originated as Syrian aristocrats and their main power-base was in Syria, so it is only natural that Syria should be central to their architectural ambitions. The other part of the answer, though, is the ongoing legacy of Byzantium, from whom Syria had only recently been conquered and who still had close ties to Syria(for example, The Umayyads in particular were very keen to appropriate the Byzantine imperial ideology of divine legitimation and the broader trappings of late antique imperium, and this produced a lot of buildings and artworks that attempted to recast Byzantine power in Islamic form.
This is most visible in the architecture and decoration of the Umayyad desert palaces but we can also see evidence of it in the Umayyad mosque of Damascus. The buildings depicted mosaics of the courtyard, for instance are quite close both to imperial Roman architectural paintings and to late antique architectural motifs(compare them with for example Pompeian wall painting or even the 'tempietto' in the Echmiadzin Gospels) and the floral motifs in these mosaics also comes from late Roman and Byzantine art, especially the use of the vine as a metaphor for the fruits of paradise that also had imperial and courtly associations. The elevation and layout of the mosque also produce similar associations. The courtyard facade's central entrance echos Roman architecture, especially the use of the pediment, and the interior arcades(arches supported on columns) also call to mind the typical late antique basilica(see for example Old St. Peter's for comparison).
Sources and further reading:
Garth Fowden, Qusayr 'Amra: Art and the Umayyad Elite in Late Antique Syria
K.A.C. Creswell, A Short Account of Early Muslim Architecture.
Finbar Barry Flood, The Great Mosque of Damascus: Studies on The Making of an Umayyad Visual Culture.