I can't speak to rare earths, but people could and did dig up uranium and thorium, but there is no way that they would have known they were radioactive. The amount of radioactivity is too low to be detectable without specific instruments (which would not be developed until the 20th century) and though some kinds of radioactive environments could have negative health impacts, these health impacts are chronic and only visible in populations monitored to some degree over time. The earliest record we have of people doing that comes from the 16th century, when it was noticed that miners in a specific region with a lot of uranium-bearing ores had a higher level of lung disease (it was later determined to be lung cancer) than the general population. But it was not clear until much later why that was the case (radon exposure).

Anyway, there are more details on this here, but the fundamental point is that anything on this planet radioactive-enough to be obviously radioactive (glowing, very dangerous, etc.) was produced or refined artificially. Such things did not exist for humans until the 20th century.