Language: The British spoke Proto-Brythonic at the time of the Anglo-Saxon invasion. It was a P-Celtic language that later turned into Welsh, Cornish, Breton, and Cumbric. We don't know what the Picts spoke for sure, but it was probably closely related. The Irish spoke Primitive Irish around the same time. It eventually evolved into Old, Middle, and Modern Irish. Settlers from Ireland brought Irish into Scotland and the Isle of Man, where Scottish Gaelic and Manx Gaelic developed. The Gaelic speakings areas made up a large dialect continuum and cultural sphere up until the Flight of the Earls in 1607.

Religion: The British and Irish would have practiced various traditions of Celtic polytheism, and possibly some form of Roman polytheism in Southern Britain. Ireland was Christianized fairly early, and Irish missionaries helped to convert the Picts. The Anglo-Saxons were pagans when they came to Britain, and were later converted by Irish and Frankish missionaries. Christianity in Britain and Ireland was out of touch with Rome, while the Franks had much closer ties. The Anglo-Saxons weren't sure which 'brand' of Christianity to go with, but they decided to eschew Celtic Christianity at the Synod of Whitby.

Art: What we consider 'Celtic' knot-work today was also used by the Anglo-Saxons. The proper term is Insular Art. We even see similar motifs in Scandinavian artwork. I'm sure an expert could identify the region and general time-frame a certain piece came from, but to most eyes it's all very similar. The font used in Irish texts up until the 1930s or '40s (and you'll still see it on pub signs) is called the Cló Gaelach. But we also find it in Anglo-Saxon manuscripts, where we refer to it as Insular Minuscule.

Housing: Celtic Britains tended to live in round huts. The Anglo-Saxons built rectangular structures.

Music: We can only take educated guesses at what either culture's music sounded like. Most of what we think of as Celtic music has its origins in the 1500s or later.

Poetry: The Anglo-Saxons maintained the Germanic alliterative poetic tradition from the continent (Beowulf), though it was beginning to die out after they were converted. It was generally thought of as pagan until a poet named Caedmon (whose name is actually British, not English) was credited with bringing back the tradition as a way to praise God. I don't know as much experience with the Celtic poetic traditions, but they would have likely have had similar subject matter. Heroes, battles, great deeds, undying fame, and all that. Much of what has survived of the Brythonic tradition was composed in The Old North (Elmet, Rheged, Strathclyde, Gododdin) and preserved in Wales. Old Irish has more literature than anyone can tackle in a lifetime, but one of the most famous stories is the Táin. Though it's prose, we can see that the Irish, British, and Anglo-Saxons weren't too far from the Proto Indo-European tradition.