No, that comic is just flat out inaccurate, likely repeating another "interesting fact" about the Norse. Establishing paternity was absolutely vital in Norse culture (as in almost any other culture that I can think of). There were laws in Iceland that allowed women to be tortured if they refused to give up the identity of the father of a child born out of wedlock, certain men had the right to refuse any child brought before them and the child would then likely have been intentionally exposed to the elements to die (unclear if this is due to social status, only in theory or if there was reasonable suspicion of adultery, etc...).

In addition, adultery was quite a serious crime in Norse societies, before and after conversion to Christianity. Penalties could include execution, so the idea that women could freely have sex and their husbands would just be ok with any children produced from such unions is just ridiculous.

Further reading:

Jenny Jochens, Women in Old Norse Society

Total aside, one day people will stop repeating ridiculous stories about the Norse and their approaches to gender, sex, and sexuality. it is quite odd how the Norse seem to have developed a following saying that they were particularly "ahead of their time" for these issues.