"Complicated" might be a good term to describe the relations between those two empires. When the Umayyads came to power in 661 they fell heir to the wars against Byzantium that had been started by the first caliphs when they invaded Palestine, Syria and Egypt. They continiued the muslim expansion into northern Africa by conquering Carthage in 698, pushed into Anatolia and even besieged Constantinople twice. Nevertheless there certainly wasn't constant war. The two powers could cooperate in certain matters. For example in a peace settlement in 685 both sides agreed to share the revenues of the isle of Cyprus equally between them. When in 707 al-Walid I. replaced the old Prophet's Mosque in Medina by a new structure the byzantine emperor sent Greek craftsmen to help with the building process. The constriction of other Umayyad monuments like the Great Mosque of Damascus or the Dome of the Rock in Jerusalem also seem to have involved byzantine artisans. So the long history of cooperation between Byzantium and its muslim neighbors already began in Umayyad times.

What was life like then in The Byzantine Empire with most of its territories

The Umayyad Caliphate and Byzantine empire were in pretty much a constant state of war with each other (mostly resulting in the Umayyads gobbling up huge amounts of Byzantine territory). The borders only became somewhat settled, just before the Abbasids overthrew the Umayyads. So at least at the level of governments, essentially, they must have had fairly poor relations.