The only reason he wanted to rule Skyrim was so he could kick all non-norse people "back to there own lands" as he would say.