We don't have anything like the sort of sources that would allow us to be sure that the relationship between Malcolm and Tostig was a "friendship," in the modern meaning of the term – but then it's unlikely either man conceived of, or had any interest in seeing, their interactions in such terms.

The basis of any relationship that the two had would have had its basis in politics. When in 1065 Harold II and the Northumbrian nobility deposed Tostig and had him declared an outlaw, he was forced to flee the earldom - going first to Flanders and then, after further military reverses incurred in the course of attempts to raid England's North Sea coast, to Scotland. He spent time at Malcolm's court, and was able to communicate with the Norwegian king, Harold Hardrada, and plan the recovery of Northumbria with him.

So Malcolm was offering shelter not to a current earl, and hence - as you point out - a current enemy - but to a deposed earl whom he could hope to turn into a future grateful client. Helping to restore Tostig to his earldom would create a bond of obligation that the Scottish king could hope to exploit.

Tostig did spend time at Malcolm's court and it's perfectly possible that the two men did discover they had things in common, but it's not necessary to posit Malcolm's assistance was based on friendship - political expediency was almost certainly a more important motive.