British MP Jacob Rees-Mogg recently said:
It is well known that Christ was taken by Joseph of Arimathea to Glastonbury, so why in 'Jerusalem' could anyone want to sing 'did', when we know that the truth is that Christ not only went to Glastonbury but, as in that old Somerset saying to assert the truth of anything, 'As Christ was in Priddy,' Christ also went to Priddy, and as a young man Joseph of Arimathea probably did too.
Is this well known? What evidence is there for this story, and when did the story first appear?
There is virtually no chance whatsoever that it's true. No mention of it appears before the 13th century, Christian writers only said that Christians had come to Britain by the 2nd century.
However, in the 12th and 13th centuries, various legends of Joseph of Aramathea associated with Britain, particularly with Glastonbury and the Holy Grail developed. In some versions of these legends, Joseph came to Cornwall with his nephew, the young Jesus*, and visited Glastonbury. This isn't the common version of the story though: in more common Arthurian tales, Joseph or his followers or his son took the Holy Grail (possibly with some of Jesus's blood) to Britain. So, essentially, it's part of Arthurian legend, and an attempt by various British writers to tie their nation into Christian legend.
*Joseph of Aramathea was almost certainly not the uncle of Jesus, it's just that in this legend he was.