The only verified pre-Columbian European contact with North America was the Vinland settlement. There are rumours and myths of Irish, Welsh, Phoenician, Egyptian and many other sailors reaching the continent from Europe or Africa before the Vinlanders, but there is no evidence to support any of them beyond a few ambiguous artifacts which are usually dismissed by scholarly historians.

While Columbus did not "discover" America, his journey was more important than that of the Vinlanders for one important reason - his trip initiated the large scale exchange of resources, culture and people across the Atlantic that continues today. We don't know exactly how much exchange there was between the Vinland colony and the rest of Europe, but we know it must have been much smaller than it was in the immediate post-Columbus era, because of the paucity of evidence.

So, while Columbus was not first, he is an extremely important figure in the history of relations between the Americas and Eurasia-Africa, so I think it's justifiable that he is given priority over the Vinlanders in history classes. Not that the Vinland settlement isn't interesting, but it is ultimately less important to the course of global history.

Source: Atlantic History: A Critical Appraisal, by Greene and Morgan.