Is there any historical proof to this allusion?
Well, it's true that Vitruvius and Pliny claim that caryatid statues and reliefs were depictions of the women of Karyai who were captured after the city was destroyed for siding with the Persians during Xerxes' invasion in 480 BCE.
They're late sources, they're not known for their reliability, they don't cite their own sources, they're the only sources who report that story, and most importantly, we know that caryatids weren't called "caryatids" in the 400s BCE; they were called korai, girls. So the balance of probabilities is certainly against it.
Edit. Oh, and there are 6th-century caryatids. Which predate the destruction of Karyai. So no. But it's true that people in the Roman imperial era believed this story.