Both Anglo-Saxons and Romans recognized that time had a lot of depth, though precisely how old ruins were, and who built them, wasn't always clear. Some Roman authors like Lucretius had developed theories of human progress from stone tools to bronze to iron, and Babylonians and Egyptians both argued that the world was tens or even hundreds of thousands of years old. There was a lot of room in the classical imagination for ancient and forgotten civilizations to leave mysterious ruins behind.
The evidence we have from Anglo-Saxon England suggests that their ideas about prehistoric ruins were less historically specific. Anglo-Saxons were very interested in ruins: about half the cemeteries we've found from the fifth through seventh centuries CE were built on or near ruins, religious shrines and pre-Christian rituals seem to have been performed in connection with ruins, weapons were given as offerings at ancient burial mounds and hill forts, palaces (like Yeavering and maybe Lyminge) were built next to ancient ruins, and by the seventh century Anglo-Saxon kings were building new graves that directly imitated ancient ruins (like Sutton Hoo). In the middle and late Anglo-Saxon periods, ruins become the meeting sites for hundred courts (local government), the foundations of churches, and increasingly execution sites and places connected with fears of witchcraft, monsters, and dangerous elves.
All of this suggests that ancient ruins were very important for Anglo-Saxons, but where did they think they came from?
Many historians have suggested that Anglo-Saxons recognized that barrows and old earthworks were built by the ancient Britons, and that families (especially Germanic immigrants) trying to prove their right to the land they lived on buried themselves near these ruins to connect themselves to the land's history. You can see some of this in Beowulf, where the builder of an ancient barrow is described with some of the same language used to describe Beowulf. The poet is clearly imagining the barrow builders (whom we know to be living in the Bronze Age in the second millennium BCE) as a clan basically just like themselves who, due to unknown misfortunes, died out sometime in the forgotten past. This belief, which while lacking in any specifics is still basically accurate, makes sense: Anglo-Saxons dug up a lot of mounds when they were burying their dead next to or inside them, and they knew there were human bodies inside them.
Many Anglo-Saxons must surely, as well, have recognized the difference between Roman and ancient ruins. The Byzantine Empire was alive and well during the Anglo-Saxon period, and there's surviving evidence of low-intensity trade between Byzantium and England during the entire Anglo-Saxon period. They must have been able to connect ruined fourth-century villas with stories of great stone Roman buildings across the sea.
But at the same time, they sometimes seem to have treated Roman and prehistoric ruins the same. At West Heslerton, a village in northern England, a ruined Roman temple was used as a shrine next to the houses of the living community, and prehistoric (Bronze Age) barrows were used as a ritual site in the nearby cemetery. They treated both spaces similarly (separating them from the contemporary spaces with fences and boundaries), but does that mean they thought they had the same histories? Or did they realize one was 2000 years older than the other? It's very difficult to say.
Some texts suggest that Anglo-Saxons didn't believe ruins to be strictly human places. The Wid Faerstica poem (a 10th century spell for curing aches and pains) talks about wild witches, elves, and the old pagan gods gathering together at barrows, and demons and monsters are often found in the same sorts of places where you encounter ruins. In most cases, it seems these monster moved in later, after the human builders of the ruins abandoned them, but an eighth-century poem (which may be about Stonehenge, which makes it the most straightforward answer to your question), says they were built by ancient giants:
Fate has shattered the wondrous, mighty stones. The city is broken, the works of giants has perished. The top parts have fallen, the high rocks tumbled, the beams are bereaved, the mortar has failed, broken holes provide no shelter from the storms; the old ones are eaten away. The worldly craftsmen, now decayed, now departed, are held in the clutch of the earth; they have rested in the grip of the grave while a hundred generations of their nation have passed away. Only the wall, lichen-covered and stained red, has outlived one kingdom after another, and remains standing against the storms, its high curves fallen.
[What their] hearts knew, their craft expressed through zeal for circle building. The foundations of the walls were wondrously supported and the surrounding earth-banks shone. There were many {standing stones} with many pinnacles, full of the sounds of war and of much banqueting and of earthly pleasures. Then swift fate changed all that. Men perished everywhere; the day of pestilence came; death took all the host of men; the warriors were stolen. The bulwarks decayed. The site was in ruins because those who should have repaired it were dead. Therefore these structures are mournful and these curved, red-stained remnants have fallen away from the circular beams. In their downfall they sank down to the ground, smashed to pieces.
Before in this place had been many men, joyous and splendidly adorned with bright gold, proud and drunk, shining in their armour, with treasure, silver worked gems, wealth, riches, pearls and the bright fortress with its wide dominion. Near where the raised stones stand there is a warm stream with a wide spring. A wall surrounds it all, and within its bright circuit were the baths, warm and ready, conveniently placed. The flow…over the grey stone warm streams…the round pool…warm…where the baths were.
An absolutely wonderful book on this topic is Sarah Semple, Perceptions of the Prehistoric (2013). It's expensive, but you can read large sections on google books. If your local university library has it, or you can interlibrary loan it, I would strongly recommend giving it a read.
Side question: was Stonehenge standing during these times? I seem to recall its being dilapidated till the 19th century.
hi! if you're interested, you'll find links to other references to Stonehenge through time in this post
There was a paper someone, I can't remember who, sent me that detailed a lot of recent archaeological evidence that showed continued occupation and use of ancient memorial sites, such as Stonehenge and various barrows. This was quite a surprise to me, an there isn't really anything I can add to the paper itself.
It is worth keeping in mind that "Roman" here means "Roman period", not necessarily people literally from Rome.