The main thing to understand about the 'vikings' is that they did not represent a unified group, and further the raids conducted by various Scandinavian elements did not represent conflict between two nation states as we would recognise them today.

My knowledge of France in this period is patchy, and this is from memory, so I'll focus on England.

The earliest raids were small undertakings, with a few ships of men raiding coastal areas, like the raid on lindisfarne in Northumbria in 793. These were not targeted attacks with a political agenda, they were somewhat opportunistic in nature.

Another important point is that neither the Scandinavian nations or England were unified, centralised countries in their own right, especially at the beginning of the viking period, and therefore couldn't draw the sort of invasion force you seem to be implying.

For perspective. The Great Heathen Army that conquered several English kingdoms in the 860's has been estimated in the low thousands, certainly never more than 4000-5000.

There are myriad reasons why England never launched an invasion of Scandinavia, not least because they didn't have the naval infrastructure, or the political unity to affect such an endeavor.

If you're interested in the subject I'd recommend Gwyn Jones' A History of the Vikings or Jonathan Clements A Brief History of the Vikings for some general overviews of the period.

To build a bit on /u/comradenewelski's post:

During the Viking period, power in France and England was really decentralized and chaotic. You can see the various kingdoms that occupied the island of Britain. Within these kingdoms, there was no cohesive, centralized military force. Kings could only raise armies through their nobles. Nobles controlled fighting forces, so getting everyone together to fight a war, paying all those people, and providing boats and supplies to send them off to Scandinavia would have been extremely difficult.

For the largest Viking invasion of England in the 9th century, civil war in the Carolingian Empire in France allowed Danish warlords to establish themselves there and make some "easy money". It took 20 years for the Franks to organize a resistance, and by then the Danish invaders had moved on to England. So many of the Vikings were based in France or England rather than Denmark and Norway.

Often small Viking bands would be paid off by local rulers, however these Viking warlords established a presence in England and France, making alliances and conquering kingdoms that had already been divided by civil war. The Vikings were very adept politically, and they found kings and lords who were willing to collaborate with them, which permitted them to continue their push for wealth.

So essentially, they had support from local English rulers and capitalized on the disorganization and chaos of the native kingdoms to push for wealth. These Vikings were largely based within England and France rather than travelling all the way from Scandinavia, so a king sending an army to Denmark would have been ineffective, expensive, and politically ill-advised.

Source is Janet L. Nelson's chapter on Anglo-Saxon England from Oxford's Illustrated History of Medieval England.

Excellent answers above and I also recommend /u/comradenewelski' s book suggestions. If it's okay I'd just like to add on a few things.

First, in addition to being disorganized, neither England (at this point it's really a group of Anglo-Saxon kingdoms permanently united under Alfred and his heirs) nor France had much of a navy to speak of. At least for the Anglo-Saxon kingdoms it wouldn't be true to say they had no experience with seafaring (there had been at least one success over the vikings that lead to the capture of longships). But it was not significant. King Alfred the Great did attempt to build a navy. And it wasn't his most significant endeavor but it did manage to have some engagements with the Norse and either win or not totally lose. But these were intended entirely to fight them in naval engagements. Not to sail to Scandinavia. Or, really, to sail much of anywhere at all.

Second, I think to say that England and France had "stronger economies" is accurate but also misleading. At least for the earliest times of the "Viking Age", it's sort of like asking who had the stronger economy in the 1980s the United States of America or Pablo Escobar's drug cartel. They are just different things. Like a cartel, at least at first, the Vikings weren't tied to a kingdom but to a leader. And they (or some of them) were enormously successful merchants and traders. The Norse had trade routes and connections through out the Baltic Sea and going toward the Eastern Roman empire and beyond into what is now Ukraine and Russia. They would sell and trade their loot from raids or spend their tribute money. But their biggest item was the slave trade. And the English had interactions with them both before and after the invasions that were entirely business dealings. The story was never quite so simple as it is made out. When the Vikings took York they devastated the area but then they revitalized it by connecting it to a broader trade network.

The mercantile nature of the Vikings gets lost in telling of them, which I think is too bad because I find it to be one of the more fascinating aspects of their culture. But they may not have had the landed wealth and agricultural resources of western Europe but they did have a vibrant and successful economic stream. I don't think it's fair to compare it because it was so different. Eventually, toward the end of the Viking Age, they did settle into established kingdoms and the follow a successful "king"/war lord model was abandoned. And, when this day came, the kingdom of Denmark was not as prosperous as the kingdom of England (and the Hanseatic league took a lot of their Baltic trade from the Scandinavians.) But that's not really how you should be thinking about the earlier Vikings (tied to one geographic area under one ruler or dependent on their land for wealth.)

Third, I think another thing to note is that if they wanted to go fight the Norse they wouldn't have to go to Denmark. Because after awhile they didn't just go away.

By the 850s the Norse had established settlements in Dublin over in Ireland. By 866 they had captured York in England. From here on is the period of the Great Heathen Army described above, which was fought of by Alfred and his heirs. But, more to my point, is they never left. Even when Alfred had beat them back they retained a stronghold in the North. And the English did make offensive moves on this land until eventually they could claim sovereignty over the Danelaw (but the settlers didn't really leave they remained). So, when the English were strong enough and centralized enough to make offensive attacks they just didn't have to go so far as Denmark or Norway (as an aside and as a generalization the people centered in Sweden tended to go East toward Constantinople rather than west toward England).

For the French, the Norse were obviously given a large bit of land, Normandy from where the other French duchies and counties and so on could wage war against them and with them for centuries to come. It took a lot longer for the crown of France to centralize the way England began to under Alfred. But, again, if they had wanted to go a hunting Vikings they didn't have to go so far.

So, I'd just sum up by saying they didn't have the centralization, power or resources to go after Scandinavia at first. Beyond that there wasn't much of a need to do so because they weren't a nation to be attacked. Once both the English or French (mostly English) had become strong enough to go on the offensive the Norse had settled a hell of a lot closer to home than Denmark.

This almost happened but didn't quite, early in the Viking Age. (I can speak to France/Frankia pretty well.) In fact, this seems like a good excuse! Let's make this a primer on Frankish-Danish relations!

The Royal Frankish Annals describe a military campaign in 808 where the Danish King, Godofrid, invades the lands of the Obodrites (a group in northern Germany) and Charlemagne sends his son Charles with an army to the Elbe to basically watch Godofrid and make sure he doesn't get any funny ideas about invading the Franks. Godofrid doesn't, although he lays waste to the Obodrites, and after he leaves Charles proceeds to wreck some other local tribes that pledged allegiance to the Danes. But the two armies never actually fight each other.

This is before large-scale attacks on France have really started. That changes in 810, when a fleet of ships under Godofrid lands in Frisia (the modern-day Dutch coast, which was Frankish land at the time) and wrecks it, beats the Frisians in battle and exacts 100 pounds of silver as tribute. Charlemagne is so pissed off at this, he sends out word for the army and navy to gather and meet him in northern Germany. It's not really clear from the translation I have if Charlemagne intended to actually invade Denmark--there was a famous fortification called the Danevirke that choked off the peninsula from the south, and this translation says that Charlemagne set up camp and "waited for what would come of King Godofrid's threats". But we never find out because while Charlemagne is waiting, Godofrid is murdered by one of his own retainers, and Charlemagne subsequently makes peace with Godofrid's successor, Hemming.

So that's the closest that the Franks ever came to actually invading Denmark. For the rest of Charlemagne's life (d. 814) and through the first part of his successor Louis the Pious's rule (d. 840), nobody's raiding Frankia, according to the Annals anyway. There's a power struggle between 814 and 823 between the sons of Godofrid and two other guys, Harald and Regenfrid. Louis grants safe harbor to Harald and leads a military campaign in 815 with the aim of putting Harald on the throne; the Danes raise an army and a navy to match his, but there's no open battle and again it ends inconclusively. Louis manages to put Harald on the throne by 821 by diplomatic means, but he's kicked out again by 826. Louis takes him in again, baptizes him, gives him Frankish land. Harald keeps trying to get back into power. A peace conference is held in 828, Harald attacks the Danes before it happens, the Danes (led by Godofrid's sons) retaliate by attacking the local Franks, and then the sons realize, oh crap, let's make peace before this gets out of hand. They do.

This is Frankish foreign policy towards the Danes. The Danes are acting like a nation-state, and they generally aren't raiding Frankish lands on their own; Danish raids are rarely recorded in the Annals before the 830s, other than the one state-sponsored raid by Godofrid, and Charlemagne responds to that by gathering an army and at least contemplating an invasion. From 814-the 830s, there's some border fighting between Harald/the sons of Godofrid, and occasionally the Franks get involved, but things are pretty okay as far as raids on Frankia itself go.

That all changes in the 830s, and it's because of civil war. The Danes weren't about to raid Frankia when it was a unified political entity, capable of raising armies and defending itself. But in 830, Louis's sons (who all rule sub-kingdoms within his empire) start rebelling against him; this leads to three civil wars between 830 and 840. Without going into pedantic detail, he was twice interred in a monastery, publicly deposed and forced to confess his crimes, and many other bad things. He kept diplomatizing and warring his way out of it, but while he was fighting his sons, the Danes/Vikings were like, "Cat's away, the mice shall play!" They started sacking trading towns and villages in Frisia and carrying off the booty, and nobody had time or energy to deal with them.

That's where the tradition of Viking raids on the Continent really begins, with the civil wars of LTP. The wars end in 840, but that same year Louis dies, and off we go again!! The sons spend three years fighting, and one of them (Lothair) actually hires Harald (yes, he's still around) to make trouble for his brothers. Harald is happy to raid and plunder, and so are other Danes. They start raiding towns and monasteries in present-day Normandy (then called Neustria). They raid Frisia. They raid Aquitaine. They go up the French river valleys (particularly the Seine and the Loire) and start raiding monasteries.

Even when the brothers' war is over, the the Vikings are still a problem, especially for Charles the Bald. Charles the Bald is king of West Francia, which is (very) roughly present-day France, and he has a hard time raising an army to deal with them. He often just pays them to go away, which works about as well as you might think it does. We're way past the 100 pounds that Godofrid extorted, by the way; the sources say that Charles was paying tributes as large as seven thousand pounds of silver, which he had to levy taxes to raise.

One really important thing to understand about Frankish politics at this time was that, as /u/theskyisnotthelimit pointed out, if you wanted an army during this time, you had to go to the nobles and ask them to call up their men. If you couldn't rely on your nobles to come fight for you, tough shit, I guess you're not going to raise an army today. Charles spent a huge portion of his time just touring the country, meeting with counts, getting oaths of loyalty from them, and otherwise making sure that his counts didn't just defect to another kingdom or rise up in rebellion. Raising an army to go fight someone else was quite beyond his ability at the time.

Most of my knowledge pertains to West Frankia (France) for the next 30 years or so, so I'll concentrate on that. Through the rest of the 840s and into the 850s and 860s, it's pretty much the same. Being Charles must have been like playing a gigantic game of whack-a-mole. He had his brother-in-law Pepin who kept rising in rebellion in Aquitaine, he had his other brother-in-law Louis the German (basically king of modern-day Germany) invade in 858 and flip a number of nobles, he had the Bretons (Brittany) to deal with, and plain old domestic rebellions or his own sons rebelling to deal with.

The Northmen, bad as they were, were only one part of a larger and uglier political picture at the time. Charles couldn't just mobilize France and go off to fight Denmark. There was no France. He was constantly using what power and influence he had to beat off the Crisis of the Day. There's at least one time in the 850s where he tries to besiege a group of Vikings on an island in his own land and he can't even do that; he has to rush off before the siege is finished. It ends amusingly if you're anyone but Charles; he ends up paying a second group of Vikings to finish the siege for him, and Group II basically waits until he's gone and then offers to let Group I leave in exchange for some of their loot, which Group I is happy to give.

So Charles spends a lot of his time running around France and chasing down various groups of Vikings, which he alternately bribes, fights, or hires for himself. It does seem like during the 860s, the Franks are able to more vigorously pursue and fight the raiders--you don't read about them getting away scot-free anymore, but rather them being brought to battle. Charles also finances the construction of fortified bridges on some of the main river valleys, so they start to choke off the influx of Vikings coming in. In the late 860s the Annals start to get really concerned with inter-kingdom politics, and don't talk much about Vikings, but even so, it seems fair to assume that the bridges and the increased political unity of the realm were having some kind of impact. The raids never really stopped, but they appear to have slowed down considerably into the 870s.

Sources:

The Royal Frankish Annals, which eventually became The Annals of St. Bertin

The Oxford Illustrated History of the Vikings, ed. Peter Sawyer

Riche, Pierre. Daily Life in the World of Charlemagne

Carolingian Chronicles: Royal Frankish Annals and Nithard's Histories, tr. Bernard Walter Scholz with Barbara Rogers.

McKitterick, Rosamond. The Frankish Kingdoms under the Carolingians