There is a thriving, legal antiquities trade involving everything from coins to expensive artwork. The key to keeping on the right side of the law if you're buying something like that is to make absolutely sure that you can verify the provenance of the item – who its last owner was, and the one before that, all the way back to its origin. However, whether this trade can ever be entirely ethical is a different question. If you accept the perspective that cultural heritage is humanity's collective property, it's hard to see how any object could ever legitimately make its way into private hands. Many "ethical" antiquities turn out to just be objects that were dug up before their country of origin had laws in place to protect its heritage, and even the most prestigious dealers and auction houses have been caught falsifying or obscuring the provenance of items.

That said, any archaeologist can tell you that there is a simply astounding amount of old crap out there. If you're just looking for something old, not particularly pretty, the earth is littered with trillions of potsherds, stone tool fragments, etc. I still wouldn't encourage you to just drive out to a field and start picking them up – as with many things, one person picking up one sherd might be harmless, but if everyone does that then an extremely valuable record of the past be gone in no time. But once it's been collected, processed and recorded a huge amount of material is hidden away in a storage facility indefinitely, or simply dumped. So if you happen to visit an archaeological site, you might ask permission to take away a few "unstratified" pieces – finds that have been removed from their original context and so are all but worthless from a scientific point of view.

However at the end of the day most archaeologists' offices I know are just full of replicas. Yes, it's nice to hold something in your hand that you know was made thousands of years ago, but you can get the same aesthetic and academic enjoyment from a replica, and you don't have to clean the dirt off them first.

Ancient coins are a big part of numismatism, and are commonly bought and sold in collector's markets. I'm not aware of any particular legal regulation regarding their purchase.