The vikings had extensive contact with mainland europeans so why didnt they bring over diseases when they established colonies around 1100AD?

As well those diseases wouldve spread and then the natives wouldve had time to recover by the time colonialism hit?

Im writing a book as well, so my scenario to explain the masive native presence is that they gained their immunity to the old world diseases once the vikings came. It sounds feasible enough for a book but when i was writing i thought why didnt this happen in real life.