The most basic reason is that 'Vikings' weren't 'Scandinavians'. Neither of these words meant the same thing then as they do mean now, and their meanings certainly didn't overlap back during the Viking Age (ca. 800-1050).

The earliest uses of the word 'viking' actually date before the 'Viking Age' and come from Anglo-Saxon England, and there's a lot of Anglo-Saxon and Frisian (Netherlands) place-names that still have variants of vik, wic, or wijk in their present form, such as Sandwich which translates to 'sandy market'. It's easy to assume that 'viking' must be derived from an Old Norse word, but since Old Norse wasn't written down until centuries later, all we know for certain is that sometime before the year 800, people speaking various languages around the North Sea could all agree that 'vikings' were one thing: pirates. (Thus commonly glossed as pirata in Latin sources; sailors were always treated like feminine nouns in Latin.)

Pirates were very scary business for the rulers of Anglo-Saxon England and Carolingian northern Europe. These were kings whose power depended on skimming the surplus off agricultural production. But when someone came in to take a piece of the pie without sticking around long enough to pick a fight, Western rulers had nowhere to turn. Especially in the rough-and-tumble kingdoms of the early middle ages, where power rested on prestige more than wealth, even a handful of rogue pirates could undermine the legitimacy of an entire kingdom.

Retaliation, then, wasn't an option. The early viking raiders could disperse without leaving much of a trace, and a counter-raid on Scandinavia would neither punish the perpetrators nor deter future attacks. A king could secure his legitimacy only by securing his territory, and in fact we see that the Carolingian and Anglo-Saxon kings trying to do just that: establishing defensive systems to deter the kinds of attacks that undermined their legitimacy.

Things got more complicated as the Viking Age progressed, since the 'viking' pirates were gradually replaced by standing armies. We've got an overarching concept of the 'Viking Age' that obscures these differences, but the people who lived through this period were quick to notice changes. The chroniclers of Western Europe had to come up with new names for these forces of occupation, such as Danes (i.e. people who spoke Old Norse) or Northmen (possibly meaning people who linked into the routes of the North-way along the coast of modern Norway; a good way to distinguish the Old Norse speakers of the North Sea from the Old Norse speakers who belonged to the emerging Danish kingdom—I warned you it got complicated!). At this point, the 'Vikings' were actually based out of Western Europe, that is, Western Europeans, despite having strong cultural ties to Scandinavia. In effect, if Alfred the Great had lashed out against Scandinavia, it wouldn't have bothered Guthrum's Danelaw at all.

So in sum, this question needs a two part answer: (1) victims of the vikings knew better than to confuse viking raiders with Scandinavian people in general, and (2) victims of the vikings knew that killing people overseas wouldn't make them any safer at home. It almost makes the warlords of the 'Dark Ages' seem downright reasonable, doesn't it?