I'm currently writing a paper on a semi similar topic to your question and I think I can provide one perspective. For the sake of precision, I'll focus primarily on Byzantine Athens. And if you're interested in further reading, there's actually a book called The Christian Parthenon by A. Kaldellis which I'd be inclined to recommend.
With that said, to put it briefly, opinions varied greatly.
Of course, there was some friction between Athens's pagan past and Byzantine/Roman Christianity. There is Tertullian's famous quote of "What has Athens to do with Jerusalem?" tied to his criticism of Greek philosophy. This derisive attitude for the non-Christian aspects of Classical Greece and Rome did persist into later periods of the Byzantine Empire. Consider this passage in the Suda, a giant encyclopedia from the 10th century. For context this is in the entry for "Adam" explaining the implications of the fall of man:
And from this point human nature became caricatured and falsified and was stamped with the shapings and designs of the Tyrant. From this source that bastard wisdom had its beginnings, for divine wisdom had made its escape and had flown up toward heaven, whence it had previously started out. Whence the Imposter expropriated the name of God and dealt it out it in many directions, giving himself different names, such as "Kronos" and "Zeus", and -- the most wicked thing of all -- the Criminal even had the gall to drag down the blessed and ineffable nature [of God] and associate it with names that were female and unworthy of respect, such as those "Rheas" and "Aphrodites" and "Athenas" and thousands of others, and into strange forms and shapes of illogical things which the Creator of Evil and the Hatcher of Heresy invented and carved out. Hence the wretched tales of the Egyptians about Osiris and Typhon and Isis, and the chicanery of the Persian Magi, and the gymnosophistry and impertinent fantasies of the Brahmans, the fabled sayings of the Skythians and the orgies of the Thracians and the flutes and Corybantes of the Phyrgians. Hence the deceitful and damaging astrology of the Chaldaeans. Hence poetry, the midwife of lies, the pretentious diction of Greek storytelling. Hence Orpheus and Homer and that portrayer of improper begettings, Hesiod. Hence the reputation of Thales and the glorious Pythagoras and Socrates the wise and Plato, the much-ballyhooed pride of the Academy of the Athenians. Hence the Parmenideses and the Protagorases and the Zenos. Hence the Stoas, and the Areopaguses and the Epicureans. Hence the dirges and breast-beatings of the tragedians and the jestings and raillery of the comics. Hence the dishonest divinations of Loxias the liar[3] and the remaining shenanigans and omen-mongering of Greek sophistication. And lest I prolong my essay by getting caught up in rotten and malodorous myths, the Imposter, having taken the burden of the entirety of creation on himself, and having taken man under his control as though he were a slave, went through all that is below heaven and patrolled the earth and kept watch over everything like a hen on her eggs, as he himself says in his lying fashion. He thought that it was necessary to set his throne above the clouds of heaven and to be equal to the Highest One.
However, the Byzantine Empire was of course heavily rooted in its classical heritage, and there were countless admirers of the classical past. Obviously, I can't give you every example of this, but I want to highlight a few quotes from Michael Choniates, the brother of the historian Niketas Choniates. Michael Choniates's texts are incredibly important because of his unique position of serving as the archbishop of Athens in the late 11th and early 12th centuries. In fact, he was the final archbishop of Athens before the 4th crusade, and ultimately surrendered the city in 1205.
Regardless, Michael was a deep admirer of Athens's classical past, but was horrified when he saw the city's contemporary state. He composed multiple speeches and poems in which he laments the current state of Athens and how its people had forgotten the classics. Unfortunately, his Greek is incredibly frustrating (just like his brother's) and some of the quotes don't have translations in English published, so I'll paste the Greek below and give you my approximation.
Michael CHONIATES Scr. Eccl. et Theol. Orationes {3080.002} Volume 1 oration 10 page 160 line 2
(160) Οὐδ’ ἐρείπιον γοῦν Ἡλιαίας ἢ Περιπάτου ἢ Λυκείου εὕροις ἂν, πλεῖστα καμών. Μόνον ἂν ἴδοις πετραῖον Ἀρείου Πάγου γεώ λοφον, οὐδὲν ὅ φασιν ἱερὸν, ὅτι μὴ στεφάνην πέτρας ψιλὴν καὶ μόνῳ τῷ σεμνῷ γνωριζομένην ὀνόματι·
You would not see even a ruin of the Heliaia or the Peripatos or the Lykeion, having mostly been worn down. You would look upon the Areopagus and see that it is only a rocky hill of bare earth. There is nothing there which they might call holy, since it is only a bare crown of rock, known only by its holy name.
To hammer this down, Michael was the archbishop of Athens and would have seen these locations in person, and his texts give a really valuable view into Byzantine Athens. And while these monuments have all supposedly disappeared, there is still a massive sense of reverence to these classical Athenian buildings.
But although what he saw of Athens was a massive disappointment, he still maintained a great admiration for the city and the Parthenon in particular. For this I have a quote that has been fortunately translated into the English which I'll put below. This is from his Letters 63.2
Athens, this ancient city, once prosperous, an enemy of tyrants, and a friendly home to wise men from all around; for this last quality it should endear itself to you, if to anyone. Let me add later glories [i.e. Christian ones] to those that are older and already famous. The city boasts a patron protector in the Queen of all, the Parthenos Mother of God, whose divine chamber here may in fact stand on the ancient Akropolis but its head actually touches heaven, or rather it itself is the edge of heaven, being the divine Parthenon of the Theotokos, a world-transcending chamber that projects a myriad of spiritual energies to those who draw near and perform the sacraments.
This quote is especially important. While Michael's perspective on the Parthenon is that of a converted Christian church, with his references to the Virgin, the object of his admiration in the passage is toward the Parthenon itself instead of the Virgin. I'll include Kaldellis's analysis of the quote because he does it better than I can.
Michael Choniates is the first known worshiper of the Parthenon. There is nothing in classical antiquity to match the intensity of his devotion, but there is something odd about it from the Byzantine point of view too, for it is directed at the spiritual qualities of the temple as much (or more) than at the holy person venerated there.
To sum it up, Michael's love for the Parthenon is really unique because most versions of this kind of veneration would have been dedicated to the Christian figure represented, rather than the object or building associated with it. In the case of the Parthenon, it would have been the Virgin Mary, but again, Michael's focus is on the Parthenon itself.
While I've only provided two examples, one being a rather extreme albeit highly fascinating source, I want to stress that from the Christian perspective, views toward the non-Christian aspects of the Byzantine/Roman/Classical past were highly complex and were never monolithic. Nevertheless, various frictions and compatibilities between Byzantine Christianity and Classical Greece were very present.