There were certainly different accents across the provinces! Catullus, for example, uses some specifically Northern Italian words rather than their "standard" Latin equivalents -- basium instead of osculum (both mean "kiss," as e.g. in Catullus 5) -- though in poetry of course this may be due to metrical rather than dialectical reasons. IIRC, this also occurs in Apuleius (a Numidian) and Petronius, both authors writing in supposedly more "common" registers to reflect the relatively lower class of their characters, as well as in graffiti from places like Pompeii and Herculaneum, where we can see words spelled "wrong" in reflection of what might have been local pronunciation. We can even see this as late as the Strategikon of the Emperor Maurice (late 6th c.): the book itself is in Greek, but the soldiers' commands are in the Balkan Latin Maurice's soldiers would have understood, quite different from what you might expect in Classical Latin. Because Latin orthography was never really standardized, or at least not in the same way modern languages are, we can see quite a few different regional accents and dialects across the Roman empire, which eventually evolved into the modern Romance languages. These languages, in turn, sometimes preserve the unusual/regional forms far more consistently than the "classical" equivalent -- there's no Romance descendant of osculum, at least that I know of, but basium became the Spanish beso, Italian bacio, French baise, and so on.

The best book on the sort of provincial Latin you're asking about is probably James Clackson and Geoffrey Horrocks' Blackwell History of the Latin Language (Malden, MA: Blackwell, 2007), especially Chapters 6-8 (p. 183-304) on the Imperial/Late Antique language. This is what I used when I took History of Latin, so it might be more suited to a classicist audience; it's pretty dense, and does not shy away from heavy linguistic lifting, and it definitely assumes a knowledge of Latin. Unfortunately, I myself don't use Latin often enough any more to know specifics outside of a few outstanding cases, which I mentioned above. If our classicists have any more current citations, they'd be much appreciated!

While I don't know the complete answer to your question exactly I do remember having a discussion with my Classics professor relating somewhat to language and some Emperors. The prime example I remember was that of Emperor Septimius Severus (ruling from 193-211) and how he fluently spoke both Punic (a dialect descended from the ancient Canaanite/Phoenician language) as well as Latin and maybe Greek as well. However he was of Punic descent on his father's side (lots of Phoenician/Punic people's and cities integrated into the Republic/Empire post Punic Wars) and it was remarked that he had a discernable Punic accent when he spoke Latin and Greek. So at least for the case of Punic/Phoenician cities and regions, the Punic language may have stayed locally strong for a long time (Septimius Severus was born 291 years after the 3rd Punic War ended) alongside Latin. So...to the Romans who knew what it sounded like, they'd likely know if someone came from a Punic region of the Empire. Those who don't know however would at least know that someone from a different region sounded different (if they did that is...after all there are wide swaths of my country - Canada - where people sound relatively the same from Ontario to B.C., while there are areas where people sound different like in Nova Scotia to super different like in Newfoundland, so I imagine that large areas of the Empire had a pretty similar accent to very different accents in other regions). Hope this at least partially answers your question. Sorry for the lack of sources, it's largely from memory from uni

Given that all modern Romance languages are descended from local varieties of Latin, it's almost certain that different dialects would have existed by the time of the late Empire. As for dialectical variation before the fall of the Empire, we know that Saint Augustine of Hippo was born in North Africa (modern-day Algeria), and that people poked fun at him for his accent when he lived in Rome. He apparently was a native speaker of Latin (in fact, he was monolingual, having never mastered Greek), so it's highly likely that Latin had begun to split up into different dialects by the time of Augustine.