So the short answer is... it depends. I'll dive briefly into the weeds below, focusing on the United States. I am less familiar with the laws and regulations for other countries.

In the U.S. the place of origin of the ancient artifact determines the legality of your actions upon discovery. All artifacts found on state and federal land are protected, and it is illegal to collect or remove those items. This goes for projectile points/arrowheads, pottery, musket balls, basketry, or anything made by human hands. Archaeologists need to see that item in context, and removing the priceless piece of history from its resting place ruins the vast majority of information we can determine about that artifact. If you do come across something that looks important on state or federal land snap a picture, write down the coordinates, and send the information to the state archaeologist or park service.

In the U.S. the restrictions, and subsequent penalties, increase when trafficking indigenous remains and associated cultural materials. Human in remains in general (yes, some awful people collect human remains) are protected from removal from public or tribal land, and in the United States remains from indigenous peoples specifically are protected by the Native American Graves Protection and Repatriation Act. Criminal prosecution for NAGPRA violations covers the sale, purchase, and transport of human remains, and associated cultural items. So, hypothetically, if you came across a looted basket at a garage sale that was part of a mortuary assemblage you would be legally obligated to try to repatriate the materials. I was personally part of an effort to repatriate several looted mummified human remains back to their respective nations, and can verify the FBI takes these cases very seriously.

Now, if you find something on private land (not your own) you need to either have prior permission from the land owner to remove that material, or you need to notify them once you discover the priceless piece of history. It is still illegal to disturb and traffic illegally exhumed human remains on private land, but the restrictions for artifacts are much less stringent. The perspective of the Society for American Archaeology, and other major anthropological associations, is please consult your state archaeologist if you uncover anything that might be a priceless piece of history on your land. Snap a picture, jot down the coordinates, and send the information to the office so they can determine the potential need for salvage work. We stringently hope you do not try to excavate the material on your own, thereby losing all the context archaeologists so desperately need, and looting materials for sale only further encourages the destruction of important sites. Donating found priceless archaeological materials at a garage sale ensures they can be preserved by experts, enjoyed by all, and studied by exhausted grad students. Repatriating indigenous remains, or associated cultural materials, is just one step in correcting centuries of genocidal erasure of indigenous history and reconciling with a violent past.

It depends on what kind of artifact you have, where it is from, and where you are. Most countries have some sort of law that protects items of historical or cultural value found in that country. These laws usually define the age, value, and/or type of items that are protected. They also specify what items may be privately owned and which can be exported. Greece, for example, protects “all cultural objects (artifacts and monuments) back to prehistoric, ancient, Byzantine and post-Byzantine times up to 1830.“ In addition, many countries maintain treaties or agreements restricting the import of artifacts from certain places. The US has agreements in place with Honduras, El Salvador, and Nicaragua regarding the import of pre-Colombian artifacts.

Items which can be shown to have been stolen, inappropriately obtained, or inappropriately exported must be returned if you are in these countries, or if you are in one that is party to one of the above agreements. The Roman statue mentioned in the question was shown to have been stolen from a museum in Germany at the end of World War 2. It was returned to the museum, although it was allowed to be displayed in Texas for a year or so, and the woman who found it has her name on a plaque in the German museum now. A hoard of over 1600 Greek coins that had been smuggled out of Turkey to the US was returned to Turkey in 1999 after a long legal battle. The American owner of the coins had paid over $3,000,000 for them, and the Turkish government gave him a medal upon their return. The medal is probably worth a lot less than $3,000,000

Of course, making something illegal doesn’t keep it from happening. The world is flush with coins, sculptures, and artifacts that have been smuggled out of their source countries. Once items are out in the wild, it can be impossible to determine where they came from. A Roman coin could easily have been found anyplace between Scotland and North Africa. If that coin is only worth a few hundred dollars, nobody will care all that much. The Turkish hoard above included amongst the 1600 items fourteen incredibly rare Athenian dekadrachms that the Turkish government very much wanted back. Items like this that can be tracked back to an arcahaeological find in a certain place will most likely provoke a return demand. On the other hand, a common Roman terra cotta oil lamp worth $100 is unlikely to provoke the interest of authorities or anyone else. The laws above actually tend to encourage smuggling, as those who find the items are not usually compensated for their finds.

The UK has a unique and interesting solution to this problem. Under the Treasure Act of 1993, anyone who finds a possible treasure is required to report it to their local coroner. For these purposes, a potential treasure is defined as:

• All coins from the same find, if it consists of two or more coins, and as long as they are at least 300 years old when found. If they contain less than 10% gold or silver there must be at least 10 in the find for it to qualify.
• Two or more prehistoric base metal objects in association with one another
• Any individual (non-coin) find that is at least 300 years old and contains at least 10% gold or silver.
• Associated finds: any object of any material found in the same place as (or which had previously been together with) another object which is deemed treasure.
• Objects substantially made from gold or silver but are less than 300 years old, that have been deliberately hidden with the intention of recovery and whose owners or heirs are unknown.

If the find is determined to be a treasure, the finder is required to offer the items for sale to museums. The value is determined by a review board. If no museum is interested in the items, the finder can keep them and do with them as they wish. This encourages the reporting of finds, as the finder either gets paid or keeps them.

Of course, the UK is still involved in one of the great cultural property controversies of all time. The British ambassador to the Ottoman Empire in the early 1800s was a man named Thomas Bruce, the seventh Earl of Elgin. Lord Elgin was a lover of history, and was appalled at the poor condition of the Acropolis in Athens, then under Ottoman control. In 1801, he obtained permission from Ottoman officials, who cared little about anything Greek, to “to take away any pieces of stone with old inscriptions or figures thereon”. Elgin interpreted this charge broadly, and sent around half of the surviving Parthenon sculptures back to England. The sculptures were sold to the British government in 1816, and then placed under the trusteeship of the British Museum.The sculptures are one of the most prized items in the museum's collection (which is saying something - the BM has the most impressive collection of antiquites in the world, and their basement probably has the second, third, and fourth most impressive collections).

In 1832, the Greeks gained independence from the Ottomans, and began to seek out cultural artifacts that had been taken from them over the years. Among these, they asked for return of the Elgin Marbles. The British responded essentially that “Hey, we had permission to take these, and we are taking really good care of them. You can visit them whenever you like. Admission to the museum is even free.” The fight over the possession of the sculptures is ongoing. (Note that neither Elgin nor the British in general can be viewed as complete villains in this episode. If Elgin hadn't moved the sculptures away it is highly likely that all or most of them would have been heavily damaged or destroyed subsequently.)

So the moral of the story is that if you have an artifact of little value, it is unlikely anyone will care. If you have an item that rightfully belongs in a museum (or came from one), someone may well ask for it back, and you might have to comply. If you have a ridiculously valuable treasure of great significance, you might get to keep it, as long as you are a sovereign nation yourself.

Reference

UNESCO database of cultural property laws: https://en.unesco.org/cultnatlaws

Article on the return of the Turkish hoard: https://archive.archaeology.org/9905/newsbriefs/hoard.html

UK Treasure Act of 1996: https://www.legislation.gov.uk/ukpga/1996/24/contents