Well who says they didn't know of sailing? I mean that literally, I've not seen it suggested.

Scandinavia and north Germany enter the 'Roman Iron Age' around the first century BCE. This is marked by a cultural shift due to contact with the Romans, with Germanic peoples serving in the Roman armies and trading with the Romans. Roman artifacts start turning up in Scandinavia. Roman style agricultural tools are introduced, such as long curved scythes like we have today (in the pre-Roman Iron Age, they'd had short broad-bladed ones). The runic writing system is created after influence/inspiration from Latin, Italic and/or Greek alphabets (in about that order of plausibility). And so on. The Roman author Tacitus also writes (c 100 CE) in Germania about the Suiones (probably Swedes) symmetrical ships that they rowed rather than sailed. So it seems implausible that people from all over Scandinavia wouldn't have at least seen Roman and other sailing vessels by that point in time.

So the question is rather: Why did they stick with rowing as long as they did?

No definite answer can be given to that (yet at least). We have no written sources until a good 5-600 years after the switch. No inscriptions saying anything about it. What we do have, is picture stones from Gotland from the 7th and 8th centuries and later which consistently depict ships with sails (and for the record; with square or rhombic patterns on the sails, not the striped sails that are a trope of Viking fiction). Preserved ships are even rarer; we're talking about essentially a half-dozen ships from the entire Nordic Iron Age that are whole enough to say much at all. The Kvalsund ship is a rowing vessel though and dated to 690 (give or take a half century) while the oldest ship with traces of rigging is the Oseberg ship from the start of the Viking Age (first third of the 800s).

The 1980s was very much the decade of the reconstructed Viking Age ship, as a number of them were made then, and experimental archaeology on sailing them lead to a lot of new knowledge and books being published about it in that time. This article (in Swedish) is a bit old but still relevant as it asks some of the main experts on the topic what their guesses are.

Professor of Archaeology and (at the time) head of the Viking Ship Museum in Oslo, Arne Emil Christensen, puts it bluntly: "Real men row." and opines that cultural conservatism could've been the reason; sailing was perhaps seen as something lazy and weak. (Until the taboo was broken, after which everyone scrambled in an arms race to build the best sailing ships they could)

Björn Varenius (head of the naval museum in Gothenburg) agrees there's something to that, but he and Ole Crumlin-Pedersen (founder of the Viking Ship Museum and Centre for Maritime Archaeology in Roskilde) point to another form of conservatism; namely that there'd be a sense that sails weren't needed and in any case difficult to implement. As the ships being built had been refined over the centuries to very fast and capable rowing vessels. And the fast rowing vessels with light keels could not just be modified to add a mast and sail; it required changes to the fundamental design of the ships.

I'd add that they were not necessarily slower than sailing vessels. On the contrary; they could travel upwind by three times as fast by rowing compared to tacking against the wind with sails. (and once they did convert to sails, the sagas seem to indicate a not-insignificant amount of time was spent waiting for the right winds) The main benefit of sails is conserving effort and allowing you to go futher.

In place-names you have the Roden/Roslagen, a coastal area of Sweden from which many Vikings hailed. That region is literally named for 'rowers', which in turn likely gave its name to the Rus'. So that gives an indication that rowing was very much part of their self-identity, for at least some.

As that article and others also point out, there's not exactly a straightforward progression to better and better boats; there were changes in fashion and priorities. The Medieval cog was in many respects a worse ship than the Viking Age longship or knarr. But it was more defensible and could take more cargo.

Could you name these sources that speak of a sudden arrival of sails? It seems like a strange assertion. Even if you assume the Kvalsund and Oseberg ships are representative for their time periods (a pretty big assumption off a single data point!) it still leaves a full century for the transition to occur. There's just not enough preserved ships to say when the transition occurred, much less if it was gradual or sudden. In any case it's not likely it was due to the sudden realization sails existed.