I'm very suspicious every time I encounter a sliver of bone claiming to be from Peter or Paul, but there are many less important relics whose authenticity is well established.

By way of example, we have a long list of texts describing various times when the tomb of St. Cuthbert, a Northumbrian (English) monk from the 7th century, was opened and its contents seen and recorded. The relics his tomb contained still survive as described, and you can see them if you go to Durham and visit the saint's tomb (and the neighboring museum). There are many similar cases where there's little doubt as to a relic's authenticity. And there are many others where the relics appear to come from the right time and place, and certainly could be authentic even if there's less documentation to show a clear history of their provenance.

The many surviving pieces of Jesus' foreskin, of course, are rather more suspicious!

There are plenty of relics still in existence, mostly of very dubious provenance.

Off the top of my head, the Topkapi Palace in Istanbul has things like Abraham's saucepan, Moses's staff, Mohammed's beard hair, a fragment of John the Baptist's skull, and others. St. Vitus in Prague has the reliquaries of St. John of Nepomuk and St. Wenceslas. The Hofburg Museum in Vienna has lots of fragments of the true cross, various saints' bone pieces, the piece of cloth that Veronica wiped Jesus's face with, a fragment of Jesus's loincloth, and the nail that nailed Jesus's left (right?) hand to the cross. Budapest has the hand of St. Steven in St. Steven's. Chartres Cathedral has Mary's veil. St. James is supposed to be in a reliquary in Santiago de Compostela.

Obviously it takes some credulousness to take all of these things at face value. That said, they were (and continue to be) things that religious people actually venerate and have in some cases for centuries.

The actual hand of St Steven I can be found in St. Stephen's Basilica in Budapest, Hungary. He was the first King of Hungary (c. 975–1038). I only saw it when travelling so if you want more details wikipedia should suffice :)