Examples include Mithral Hall from Forgotten Realms, the Lonely Mountain from The Hobbit, Tronjheim from Eragon (not to be confused with the similarly-named Norwegian city), etc. I'm specifically not referring to civilizations on the slopes of mountains, like the definitely-not-dwarf Qanuc of Memory, Sorrow and Thorn - they need to carve great cities within the mountains themselves.
There are a couple of really neat strands of history/folklore behind the 20th/21st century fantasy genre!
As you might expect from Tolkien the medievalist, the first source is medieval literature. In this case, Old Norse literature (and whatever earlier traditions it may have drawn on, although we can't know for sure). Dwarves in particular are almost always associated with living in or near stone, often straight-up inside mountains--Werner Schäfke counted 22 out of 29 cases. Giving the impression we have of larger and more elaborate carved-out caves, the dwarves' dwellings are also often linked with being large and beautiful.
ETA: Oh! I almost forgot: after the Norse sagas, the "tale grew in the telling," as it were. Dwarves became associated in particular with mining, which strengthens the association between dwarves and mountain-dwelling. This development is not entirely as innocent as it seems, though. Stereotypes of dwarves and stereotypes of Jews, in the European Christian mind, began to weave together after the Middle Ages. So the idea of gold and greed gets woven into the legends, too. (This is a radically simplified version; I have an earlier answer on this, but I can't find it right now.)
A second major association is the legend of the sleeping king--a mostly-dead hero who is...sleeping...inside a mountain, awaiting the proper time for his triumphant return. It's not always a full-blown mountain here. For example, Merlin in the Arthurian legend often ends up trapped inside a rock forever. (In some versions, it's ambiguous as to whether Nimue might have stuck him in a tree, because the general story is that she traps him while he's resting against a tree in a forest.)
But the more famous version is the actual king who sleeps. I study medieval Germany, so unsurprisingly, I'm most familiar with the legend as applied to 12C Holy Roman Emperor Frederick Barbarossa. But it's very, very widespread.
The sleeping hero legend/folklore motif isn't a civilization, of course. But it nevertheless captures our imaginations about what, exactly, lurks in the inner mountain darkness.