Yes. They thought the great palaces at Mycenae and Tiryns had been built by giants in the heroic age.

Ancient history covers a really long period. It's often pointed out that Cleopatra lived closer to our time than to the time the Pyramids were built. Even a thousand years before Cleopatra, there were already people in Egypt whose job was to study the nature and purpose of the Pyramids. Literally, their civilization was so old that its early achievements had become a mystery to the Egyptians themselves. Something similar apparently happened to the Greeks; over time, real knowledge of the Mycenaen period was lost, and the ruins that remained became the seed of legend and speculation.

Also, there's a scene from Xenophon's Anabasis in which Greek mercenaries encounter an ancient, abandoned city in the heart of the Persian Empire. This was probably Nineveh, the capital of the Neo-Assyrian Empire. Xenophon marvels at the size and circumference of the walls, and notes that some locals have taken refuge from the invading army on the flat top of a pyramid (clearly a crumbling ziggurat).