I know that there was some Christian religious literature produced in the late Empire that was written in Syriac and also Coptic, but were there any other works, especially of a non-religious character?
Yes; local dialects survive poorly but are accounted for. One vivid example is 'Oscan', the language of Pompeian people before the capture of the town by Rome in the early first century BCE. Graffiti written in Oscan survives in the walls, some of it directing town defenders to points of interest. It was also spoken throughout southern pre-Roman hegemony Italy. It was alphabetacised Ind survives in inscriptions, but Italy's climate prevents survival of most organic writing materials with exception of select circumstances.
After the capture of the town and the settling of a Roman colony in it, the language seems to have declined dramatically. On the plus side, the Oscan speakers seem to have really enjoyed new Roman customs like baths and the games. Shame they were all exterminated by Vesuvius a century later.
Further reading; Minor, K. Graffiti and the Literary Landscape in Roman Pompeii. OUP 2014.
Ovid (1st century BCE-CE) claims to have written a poem in the Getic language while exiled at Tomis on the Black sea (Ex Ponto 4.13 - Loeb translation):
"Nor should you wonder if my verse prove faulty, for I am almost a Getic poet. Ah! It brings me shame! I have even written a poem in the Getic tongue, setting barbarian words to our measures: I even found favour - congratulate me! - and began to achieve among the uncivilized Getae the name of poet. You ask my theme? You would praise it: I sang of Caesar."
Now, Ovid writes this in a (Latin) poem, and in earlier poetry he specifically warns against identifying the persona of his poetry with the man himself. So, we don't actually know whether or not he really did write in Getic.