There is an episode from the later 3rd century which does bear a certain resemblance to the events you describe. During the reign of the emperor Probus (r. 276-282) we hear of some seaborne raiders causing no end of trouble across the Mediterranean. Probus had forcibly resettled some troublesome Franks on the Black Sea coast to the north of Greece but this did not pacify them. They staged an uprising and were able to gain control of a squadron of ships and sailed south toward the Bosporus. They made it through the Marmara Sea and into the Mediterranean where they set about plundering coastal communities in Greece, Anatolia, and even as far south as Libya. They must have been quite effective warriors as they are mentioned as having sacked Syracuse at one point. They were repulsed at Carthage but without serious losses. Their Mediterranean adventure over, these daring Franks went through the Straits of Gibraltar and apparently turned north with the intention of returning to their homeland.

This all speaks to the chaos afflicting the empire during the Crisis of the 3rd Century, seeing as the Franks were able to seemingly plunder with impunity. In terms of which sources describe these events, I am only familiar with two but there could certainly be more. The Panegyrici Latini is a written collection of anonymous orations describing the reigns of various Roman Emperors in the late 3rd and early 4th centuries and it makes a brief mention of this great Frankish expedition. The other reference is found in Zosimus, a Byzantine historian of the late 5th and early 6th centuries and is similarly brief. The battle at Carthage is mentioned only in this narrative.

See here for the Panegyrici and here for Zosimus. Admittedly these events happened more than 20 years after the time in which your novel is set and the pirates were Franks rather than Goths so the two stories aren't entirely the same. But there are some distinct similarities which make the story in the novel reasonably plausible.