So far as I know they usually were carried in the warriors hands: at least I can't think of an example of another way they were carried.

For instance Giraldus Cambrensis, in his 1188 Topographia Hibernica (some time after the viking period, but he says that the use of the axe was borrowed from the Norwegians and the Ostmen, and this is backed up by archaeology) says that the Irish:

always carry an axe in their hands instead of a staff, that they may be ready promptly to execute whatever iniquity their minds suggest. Wherever they go they carry this weapon with them, and watching their opportunity as occasion offers, it has not to be unsheathed like as sword, nor bent like a bow, or thrust out like a spear. Raised a little, without any preparation, it deals a deadly wound. They have, therefore, always at hand, nay, in their hands, that which is sufficient to inflict death. From these axes [securibus] there is no security: while you fancy yourself secure, you will feel the axe [securim].

Now there are a few things going on here (puns, racism), but clearly Gerald considered an important attribute of the axe to be that it wasn't stored in some way, but was always ready to strike. Earlier he describes how 'in striking with the battle-axe [the Irish] use only one hand, instead of both, clasping the haft firmly and raising it above the head', but we can presume true two-handed axes would be even less likely to be worn in some way (and if they were, it wouldn't be from the belt, they're just too long).

Bonus: pictures of axes from an edition of the 'topographia' 1 2 3

edit: there's a guy on the Bayeux Tapestry carrying an axe while fighting with a spear. It's not entirely clear whether he's supposed to be carrying it in his left hand, or whether it's attached to his back somehow (though that seems less likely).