This is the quote from Constantine Porphyrogenitus, which is where we have this information from: De Arte Imperiando, 224:

The inhabitants of the district of Maina are not from the breed of the aforementioned Slavs, but are of the older Greeks, who are to this day called Hellenes (pagans) by the locals for being pagans in time past and worshippers of idols, like the Hellenes of old, and were baptised and became Christians during the reign of the late Basil (867–886). (Ἱστέον ὅτι οἱ τοῦ κάστρου τῆς Μαΐνης οἰκήτορες οὐκ εἰσὶν ἀπὸ τῆς γενεᾶς τῶν προρρηθέντων Σκλάβων, ἀλλ’ ἐκ τῶν παλαιοτέρων Ῥωμαίων, οἳ καὶ μέχρι τοῦ νῦν παρὰ τῶν ἐντοπίων Ἕλληνες προσαγορείονται διὰ τὸ ἐν τοῖς προπαλαιοῖς χρόνοις εἰδωλολάτρας εἶναι καὶ προσκυνητὰς τῶν εἰδώλων κατὰ τοὺς παλαιοὺς Ἕλληνας, οἵτινες ἐπὶ τῆς βασιλείας τοῦ ἀοιδίμου Βασιλείου βαπτισθέντες χριστιανοὶ γεγόνασιν.)

We know the late pagans lived in Laconia, the district around Sparta, and specifically in Maina, now Mani Peninsula, the inaccessible middle finger of the Peloponnese (in more senses than one), that proved such a challenge for both the Ottomans and the Modern Greek state to establish dominion over.

Porphyrogenitus reports the conversion as happening around 870; but Nikon the Metanoeite was undertaking missionary work in Byzantium while Porphyrogenitus was still writing in the 950s. He worked on rechristianising the recently reconquired Emirate of Crete, but he is most renowned for his missionary work in Maina, and he died in Sparta. Wikipedia reports that “The Maniots began to convert to Christianity in the 9th century AD, but it wasn't until 200 years later in the 11th century AD that the Maniots had fully accepted Christianity.”

That it took that long is news to me, and I don’t have a source for it. The Oxford Dictionary of Byzantium notes that missionaries were active in the 9th and 10th century. It also notes that there were already four churches built in Mani in the 5th and 6th centuries, and “there must have been considerable reversion to paganism during the time of Slavic settlement in the peninsula”. The OEB thinks it strange that the conversion is not mentioned in the Vita Basilii, the biography of Basil I within Theophanes Continuatus, which is also attributed to Porphyrogenitus.

We know the late pagans were distinct from the Slavs who also lived in the Peloponnese, and who still spoke Slavic in the region north of Mani as late as the 1430s. Unlike the Maniots, we can assume the Slavs in the Peloponnese had long since been christianised.

The dialect of Mani is archaic, and has been grouped together with the archaic dialect of Attica (including Old Athens, which died out after Athens became the capital of the new Greek state.) It is nowhere near as archaic as Tsakonian, the language spoken in Cynuria, which has some remarkable survivals from Doric. (The extent of morphological meltdown in the language is even more remarkable—but not as romantic.)

Cynuria and Mani are at the opposite ends of Laconia (although administratively Cynuria is now counted as part of Arcadia, not Laconia). Tsakonian is extremely archaic, and was if anything even more cut off from the rest of the world than Mani (at least Mani had decent harbours); as a result, linguists tend to assume the description by Porphyrogenitus encompasses the Tsakonians as well. But that’s an extrapolation.