There is really no way for us to know what an average farmer believed during the time period. So, we are at a bit of a disadvantage. For the elite Rome was never lost.

The elite were not only aware of Roman culture they were in some ways connected to it. Offa of Mercia was a frequent pen pal of Charlemagne, who would come to be named Emperor in emulation of the Roman emperors and dealt directly with Constantinople. He was quite aware of the Romans. Charlemagne's monk tutor/adviser, Alcuin, was from Northumbria and was sent back to England on Charlemagne's bidding for several years. He was a highly educated man himself and knew of Charlemagne's interaction with Rome and how he sought to emulate it. Theodore of Tarsus was Archbishop of Canterbury in the 660s. He grew up in the Byzantine Empire. He introduced a school there and made the area known for learning. The students who studied under him and his cohort, Hadrian from North Africa, would have had extensive contact with and awareness of the Byzantine Empire. And the Byzantine empire was the Roman empire. Alfred the Great traveled to Rome with his father as a child. (I'm really weaving back and forth through the centuries here but it's a large time period. Connections to Rome start early and keep going.)

Beyond that the Church was a big deal in Medieval England. Despite what people think Christianity never really left after Rome left the island as the native Britons were still largely Christian even though the tribes that came to be called the Anglo-Saxons weren't originally. But by the time of Vikings they had been converted completely for a very long time. You can't have the gospel read to you and not be aware of the Romans.

There is evidence that Edwin king of Northumbria (who converted to Christianity) implemented Roman aspects to his court. There is some evidence of an amphitheater built under him (of timber not stone so we have little). And Bede mentions that he was preceded by a standard bearer with a Roman like scepter or globe. Intentionally mimicking Roman emperors, if he did, even on a small scale certainly suggests that he thought people would be aware of the reference.

A lot of this idea, which is very prevalent, is based on an Anglo-Saxon poem called The Ruin. It's a sad reflection on ancient Roman ruins and how their current society can't match it. It includes the lines:

Wondrous is this wall-stead, wasted by fate./Battlements broken, giant’s work shattered.

So, many have taken this literally to me the author thought the ruins were the work of giants. That's a bit of a leap. It assumes the author was incapable of any sort of metaphorical thinking. Beyond that even if a poet thought so it doesn't mean it was a prevalent belief. I can find books saying Lizard People are in control of the government. Doesn't mean it is a popular belief.

So we sadly can't know what everyone thought. It certainly isn't impossible that in the countryside peasants spun tales of giants. It's also not impossible that even if they did they didn't believe them. People make up stories to entertain ourselves and others. But we know that the elite knew of Rome in the broad strokes and that they had been in Britain at one point. And we don't really have any strong evidence that peasants thought otherwise.