This is more a question of (1) linguistics and (2) prehistory - you might have luck on r/linguistics as well.

It is difficult to say what forces caused language spread if we go back to even late prehistory, a period when it’s hard enough to figure out what language families there were to begin with.

One caveat to add is that - very recent immigration aside - Norway does have one other language represented for centuries: a variety of Romani (technically very distantly related to Norwegian).

But a few factors can be pointed to as considerations. First, what is now Norway has long been cold, and second, a fringe.

Being cold means it has not been easily able to sustain a large population density, and up in the Norwegian mountains it has generally been even colder, not an ideal climate for many self-sustaining ‘hill tribes’. In the Arctic, where the Sami long predominated, it’s colder still.

Meanwhile, the Caucasus is not only warmer, but (acknowledging the Black Sea) basically in the middle of a large Eurasian continent - a wide region from Anatolia to Egypt and Iraq that saw great technological and agricultural development, as well as trade and conquest, for most of the last 10,000+ years - for most of which Scandinavia was purely inhabited by various hunter-gatherer cultures. Meanwhile Shulaveri-Shomu and Kursk-Araxes and Maykop cultures of the Caucasus attained a relatively high level of development and interaction with a linguistically diverse Middle East (for the latter two, with its Sumerians, Hurro-Urartians, Kassites, Semitic speakers, and later Indo-Europeans). It was also on the fringe of the Steppes, which saw most of the migrations.

The three ‘indigenous’ Caucasian language families (Northwestern including Abkhaz and Kabardian, Northeastern including Chechen, Ingush and Avar, and Southern or Kartvelian including Georgian) have been there for a very long time, since prehistory, and it is hard to trace their separate origins and development. Experts on the prehistoric Caucasus can chime in more. However, groups that emerge there in the historical record have come from multiple directions: we have the Armenians, who it seems (as a linguistic group, if not necessarily most of the people themselves) must have come from the ‘Kurgan area’ to the north, as did the Russians more recently. Various Iranian groups came from the south-east, Turkic groups and the Mongols (today, the Kalmyk) came in from the east via the south… and the Caucasus was nearby - if not technically bordering - the Steppe region that was lush enough to sustain many people but not to settle them and which thus saw so many nomadic migrations, many winding up in the Caucasus either from the north or passing through Persia to the south during periods of weakness. But during most of this period Norway was at the fringe of an already IE-dominated world.

The Sami seem to descend from groups that go back around 10,000 years to the Fosna-Hansbecker and Komsa cultures, but the Uralic languages they speak today arrived only within the last 5000 years, not too long before the IE languages arrived. How this exactly happened we do not know, and we do not know how linguistically diverse the region was beforehand, but Scandinavia ended up conquered by two major groups. The lower population density may have been a major contributing factor. However, since that time, when the historical record has shed more light there, Scandinavia has been spared Steppe migrations and civilisation conquest by the buffer of related Germanic cultures.

Why are the Sami and Northern Germanic language branches not as diverse as especially the northern two Caucasian families? Well for one thing, they haven’t had as much time to diverge since they took over, partly for the reasons above.

Finally, there has to be an appeal to randomness. We like simple formulas, but diversity doesn’t simply scale with size of a contiguous region. Australia could have far more diversity and - if the northern part of the continent is any indication - across its prehistory probably has, but Pama-Nyungan languages spread to occupy most of the continent, and tellingly most of that was sparsely populated for the opposite reason. Africa doesn’t have anything like as many separate families as one would expect by scaling to, say, New Guinea, because Niger-Congo and Afro-Asiatic languages (and possibly at least major subgroups of ‘Nilo-Saharan’) spread across the continent with advances in technology relative to the groups they expanded across (though this simple versions of that cause are also not quite correct and difficult to tease out). Informally, the larger a contiguous area is, the more space for different groups to form, but the higher chance of some group reaching whatever required combination of advances or attributes enable it to snowball and take over that larger area, whether by influence or conquest. So we don’t expect it to simply scale. Finally, we have to remember that our notion of what constitutes language families not only varies by researchers but is fundamentally research-dependent: it’s probable that all spoken languages have a common ancestor in a sense, but a language family is a maximal group of languages where their relationship is still detectable by modern linguists. Much beyond 10 millennia or so, it seems that with current methods any vestiges of relationship between two languages are indistinguishable from random noise. So if we wanted to assume some natural stochastic law and, for example, two places see mass language replacement every 15,000 years on average (purely for argument’s sake!), and there happened to be a mass replacement in region A 3,000 years ago, but region B only saw one 12,000 years ago, then region B would be far more linguistically diverse right now - even if this isn’t true ‘as a law’. And of course Norway was uninhabitable in the last glacial maximum 12,000 years ago.

So we happen to be at a point in history where Indo-European languages have a common ancestor within the last 5000 years that is relatively easy to detect. Before the IE languages swept across Europe, the linguistic landscape seems to have been much more diverse, and we mustn’t extrapolate from current distributions to assume this is the norm for a particular of region. It happened to miss most of the Caucasus (apart from the Armenians, later Iranian groups like the Scythians, and now Russians) - and this being at the edge of history, we don’t know why in enough detail. It is possible that the combination of a higher level of agricultural development, a warmer climate than the far north, and the difficulty of defeating and culturally converting many remote pockets helped to preserve them. It’s perhaps fairer to say that mountainous regions can lead to more isolation and thus divergence, but doesn’t have to, and many forces are at play. Even in linguistically diverse New Guinea, with its famous Highlands, the bulk of diversity is in fact in the Lowlands, some large proportion of the Highlands speaking the ‘Trans-New-Guinea’ languages (which may have spread through very early agriculture, another major area of research and more questions than answers). So no, mountainous regions don’t automatically mean more linguistic diversity - but may allow for more.