The short answer is "we don't really know" with the addendum "we probably won't know for sure anytime soon either". It's incredibly hard to get compelling evidence when it comes to toponyms (place-names). Let me elaborate on this to the best of my knowledge.
The oldest toponyms in use are generally river/stream names. It is possible that pre-Celtic river names are still in use in Britain, but such elements are not easily identified. Just because we cannot explain a name within our knowledge of the various languages of Britain (Celtic substrate, Latin, French/Norman, and of course several Germanic languages) doesn't mean the anomaly isn't just a product of contingency. Eckwall says no river name strikes him as 'probably' pre-Celtic.
There is no consensus on this however, and other authors have argued that some names are likely to be pre-Celtic. For instance, some Scottish rivers have Indo-european names which aren't of Celtic origin. This leads Nicolaisen to affirm that they are presumably pre-Celtic. Such river names are Ale, Ayr, Farrar, Naver, Shiel...
In short, it's really hard to know for sure. One needs to exercise extreme caution when dealing with ancient toponyms (especially since there is no written record of them at the time being). While the study of place-names is an interesting (and, to my mind, fascinating) source of information, the exact nature of that information isn't always clear.
Sources