So, this is less a What If and more a How much. I'll try to frame the thought process that I'm curious over...
We know of the existence of civilisations from artefacts and we know of extinct species from fossils, but both are basically statistical samples. There are writing systems that we only know of from a few thousand characters. The number of dino bones we've found represents a tiny portion of the number of dead dinos that left bones lying around. I think it's safe to say that there have been species on the planet that we don't know about simply because no fossils of them were ever found, and it's a safe bet that there are civilisations (I use the term loosely) that were lost due to being too small, too old or simply got overwritten by newer civilisations in the same place.
So here's what I'm wondering. If Earth contained a prior civilisation, how recent and/or how advanced would that civilisation need to be for us to reasonably certain of finding evidence that it definitely was there? Or to flip the question around, is there a point at which there could have been a Roman, Middle Age or modern level of civilisation which ended, and in the period since then all traces of it have been destroyed by various events?
I want to be clear I'm not proposing that such a civilisation ever existed, nor am I questioning how long traces of our civilisation will last after we inevitably die off or leave. I'm not even proposing that sentient life existed on the planet before we thought it did. I'm curious specifically about the limits of "detection" in large scale history, and what a more informed view is on how large or advanced a species has to be for us to be able to tell that they specifically were there.
Does any of that rambling question make sense?
Try ask in /r/Archaeology and /r/AskAnthropology/. Those are the best groups to ask these type of questions.
Perhaps your question was based on this, but a recent paper discusses this exact topic. See The Silurian Hypothesis: Would it be possible to detect an industrial civilization in the geological record?. Here's a news article on the paper with a bit of background. I'll let the paper speak for itself, but generally they are suspicious of any "macro" traces, or artifacts, left after geologic periods of time, but suspect that some broader traces of global effects may remain, though detectability depends a lot on longevity.
(Personally, I expect artifacts of our civilization to be findable for quite a long time: the products of advanced industrial civilization are rather different in kind from the wholly organic remains the fossil record mostly misses. The presence in sedimentary materials of highly-refined gold, platinum, tungsten, carbon, glass, aluminum, and all sorts of other stuff would be a dead giveaway, and we have that everywhere. Not to mention artifacts on the Moon and in high orbits, which could persist for a very long time indeed. But it's a very cross-disciplinary--and speculative--question, so there's lots of room for study.)