I was told years ago that oarsmen in ancient times were strong and resilient enough for a Roman slave galley to go as fast as a modern craft. Is this fact or fiction?

There was no such thing as a Roman slave galley. Warships in the ancient world were rowed exclusively by free men, and the establishment of standing fleets in most cities coincided with the entry of the unpropertied into the political arena usually. There are only an extremely few situations, all of them during times of extreme crisis when free men could not be called up. Following Cannae not enough rowers could be found, so Titus Otacilius equipped his fleet with slaves provided from men rated over 50,000 asses. But this is not symptomatic of the slave economy, since Tiberius Gracchus at the same time raised an entire army of slaves as there was a shortage of free men. Likewise, despite similar pressures, Scipio insisted that his rowers be free men and equipped his fleet with rowers raised by the allies. Scipio impressed some of the slaves and citizens of Carthago Nova into the fleet, with promises to free them later. The only other instance I know of in all antiquity where slaves were used to row the fleet is Octavian's use of slaves before Actium--but these men were all freed before serving, so they don't really count.

I don't have my copy of Casson on hand to verify precise speeds, but ancient warships could go quite fast under oars. "Modern craft" doesn't mean anything, though, a modern yacht is much faster than a cargo ship. A trireme or other galley under oars could never go as fast as, say, a naval destroyer, but they were certainly as fast as many more modern vessels. Oars, however, were rarely used. Contrary to the popular conception (you'll notice that pretty much everything Hollywood thinks about ancient maritime warfare and sailing is patently wrong) of the galley being under oars at all times, often even power-sailing (which was unknown to the Greeks, though possibly used by the Romans, although even they took down their masts for battle), oars were only used during battle. Sailing was done almost exclusively by...well, sail. Like all sailing ships, Casson noted, the speed of a warship in antiquity would depend to a great extent on the conditions of the wind, and our sources do not generally take this into account, giving us averaged distances traveled that aren't always helpful. Generally warships and trade ships are quoted as moving somewhere between 3 and 5-6 knots under sail, according to the distances and times that we are given, depending on wind. Under oars we have distances and times calculated in the order of 8 knots or so, but we have no figures for how fast ships under oars might have gone in battle, the distances and times are simply too short for then to have been recorded. Probably in actual battle sudden spurts of speed could be reached vastly in excess of what was possible for, say, Aemilius Paulus when he reached 8 knots sailing to Corcyra, since the distances were so much shorter, but we have no way of knowing