The Vikings weren't actually big axe users. In war their primary weapon was the spear with the axe as a backup sidearm. Until the rise of firearms spears and other polearms have been the primary close combat weapons by almost every culture around the world. But why the axe was much more common during the Viking or Migration Era was primarily because swords were still relatively expensive whereas axes were much easier and cheaper to make. But in Europe the quality of steel became much greater, many more smiths had the knowledge of how to make swords, and of course swords could last longer than their owners so you had an every growing amount of them available. So that by the 12-13th Century pretty much any common soldier could afford a sword as their sidearm. In wills we have what are described as old rusty swords valued as low as 2 pence and to give you an idea of how much money this is an English foot archer in the Hundred Years War was paid 3 pence every day.

If you're just comparing a sword to an axe then the sword has many more advantages. A sword is lighter, longer, edged along it's entire length, better balanced, more nimble and can stab. The sword is just a much better weapon. If you add a shield into the equation then you overcome the defensive limitations of the axe so that it can become comparable to a sword. If the opponent is armoured then the fact that the axe has all it's weight at the end gives it more percussive force then a sword which gives it a bit of an advantage which is why they didn't completely disappear. But a mace or a warhammer is a better anti-armour option to use as a sidearm, especially against plate armour since their spiked ends can bite into the armour to penetrate it and fully deliver the impact of the blow. Also if you're in a fight with lots of armour involved then it's most likely that you will have access you your polearm with it's much longer reach and power compared to your sword, axe, mace, etc.

So the axe just really didn't excel at anything. It was always going to be inferior to polearms, the main battlefield and guard weapons. In a civilian context where armour and big cumbersome shields were much less common then the sword has a huge advantage. And a final note which may seem silly is that a sword is easier to wear and much more secure in a scabbard than an axe that's just tucked into your belt or hanging from a ring, in a battle it would be quite easy to lose an axe that way.