Eiríkr Gnúpsson was the first bishop of Greenland (with a see at Garðar), althouhg for a fairly short period. The annals of the kings of Iceland (Annales Islandorum Regii or Konungsannáll) writes:

"1121 - Bishop Eric of Greenland went in search of Vinland".

Not a whole lot of story there. Little is known about him, although there are other sources to his existence (E.g. the Law Man's Annals, Lögmannsannáll , mention him leaving Iceland in 1112) So the annal here doesn't even say if he found it. If he did, it wasn't recorded. There was no colony in Vinland. Leif Eriksson and compay had gone there in 1000. By the Vinland Sagas they didn't stay there long. Nor does the archaeological evidence at L'Anse-aux-Meadows indicate that was inhabited for a longer period (or intended to be). More of a temporary base for ship repairs and perhaps further exploration, than a colony. In any case it would've been long abandoned by 1120. Again, neither the archaeology or written account supports the idea of a permanent settlement that lasted over a century. If the annals are to be believed, Eric would've been looking for the place, not the colony.

It's worth mentioning though that these were not the only trips made west from Greenland. Written accounts mention journeys to Markland (probably the Labrador coast), perhaps to gather lumber, which existed there at the time, and Helluland (Baffin Island), the latter to trade with locals. For the latter there's now some archaeological evidence too.

When it comes to the so-called 'Maine penny' there is a lot of reason to be skeptical. First, there's a history of hoaxes here. Prior to the discovery and excavation of L'Anse in the 1960s, when the Vinland Sagas were commonly known but also commonly doubted, some zealous Scandinavian-Americans seemed to take it upon themselves to provided the missing material evidence. So you have, for instance, the Kensington Runestone from Minnesota where two Scandinavian immigrants in the late 19th century found a runestone in their backyard showing medieval Scandinavian explorers had been there. Not just a remarkable coincidence but also located a massive distance from anywhere Vikings had been known to be, and perhaps even more remarkably, it was inscribed using a 19th century runic alphabet and 19th century Scandinavian grammar and words. Yet even that is still more plausible than the case of the runes found in Oklahoma.

The 'Maine Penny', although closer, is still hundreds of miles farther from L'Anse-aux-Meadows than the latter is from Greenland. (personally I think it's notable here that Norway-Iceland-Greenland's tip-L'Anse are all about the same distance: 1200 km)

The find of the coin was not done by professional archaeologists in a controlled setting. Rather a lone Swedish-American history buff - and coin collector - found it in 1957. Subsequent excavations revealed no other Norse artifacts in the area. Nor have any coins have been found in L'Anse, in Greenland, in Baffin Island, nor very many from that era in Iceland even. On top of that, if one were to plant an artifact from a particular era, a coin is by far the easiest object to get. You can just buy them from coin dealers. (Otherwise there's not much of a market for other Norse artifacts, and the Nordic governments work to keep it that way to prevent looting of sites)

Then there's the dating of the coin, which is indeed also well after any known journey towards Vinland. The dating of the L'Anse site (989 - 1020) is in agreement with the Vinland Sagas but a coin from Olaf Kyrre would be at least half a century after that. Now one can flip that on its head and argue that if someone were to plant a coin to prove Vikings had been there, wouldn't they choose the 'correct' vintage? But the problem there is that 'correct' coins weren't common then and even less so now.. The first proper domestic coins (not counting bracteates) in Denmark and Norway weren't struck until 995. Not long before Leif Eriksson's journey (possibly even after it). They were not the most common coins in those countries even at that time. One could argue then (and it has been argued) t hat a well-informed hoaxster should plant an Anglo-Saxon penny then. (IMO, the problem here is that this would not serve the motive of proving decisively that Scandinavians were there)

So there's no shortage of issues here. Assuming that Bishop Eric not only reached Vinland but went beyond it, and brought this coin with him and lost it, only piles on the unproven guesses.

If the coin is authentic the more likely explanation would then be that it arrived there through Native American trade. Even that is a stretch though. Since there's no evidence of silver coins being traded to the Inuits. Why would they value silver coins? We know that iron and woven cloth was traded with them; practical materials they did not have.