Whether you count Basque as "pre-Latin" or not, given that it's still around and some of the borrowings are into modern Spanish in modern times, there's a number of Basque words in Spanish; izquierdo, izquierda "left-handed person," "left" come from Basque, displacing earlier siniestro, siniestra which derived from Latin. But that borrowing happened in the late medieval or early modern period. Relatedly, French has a few words that may be derived from Aquitanian, an extinct ancient relative of (possibly ancestor of) modern Basque. There are also number of French words derived from Gaulish, a Celtic language which was spoken in France before Latin got there.

Italian shows influence from languages of the Italian peninsula other than Latin, such as Osco-Umbrian and Sabine. There are also a number of words from neighboring Italic languages that made it into Latin, some of which then persisted in modern Romance languages. (The classic textbook example is classical Latin rufus "red", derived from Sabine, but the Western Romance languages inherit a form that appears to be identical to neither that nor the native Latin form ruber, rubeus; I have no idea if these derive from yet another Italic substratum.)

I suspect /r/asklinguistics/ will have a more and better examples.

One source to explore is toponymy (names of settlements and geographical features) for some reason Rivers especially have a habit of keeping their name amidst major linguistic shifts. So in my area (South West England) we still have rivers with their Celtic, Pre-Roman names like the 'Avon' (lit. the 'River') and the Severn (cognates with Latin 'Sabrina' but Celtic in origin)

Here's a wikipedia list on Celtic Toponymy with plenty of examples from Italy, Spain, France and Portugal. http://en.wikipedia.org/wiki/Celtic_toponymy

Why does everyone always forget Romanian??