This has been asked a few times before.

Here is probably the best answer:

From /u/pqvarus:

They certainly really thought that the gods lived on top of Mt Olympus, however they certainly didn't expect to actually meet one of them in person once they climbed up there. This is connected with one important feature of ancient greek religion - the interweaving between the human and the divine sphere. Gods and divine beings (apart from the Olympians there was a vast amount of other gods, spirits and demons) were or at least could always be present in every day life and had specific dwellings. In most of the cases these dwellings were prominent natural formations such as mountains, caves, rivers or groves just to name some possibilities. The home of the twelve gods on Mt Olympus is just a prime example for this case. And now let's imagine a cave that is said to be home of one or more nymphs. Of course you will never see one of them physically when you enter there - but you are nevertheless able to encounter them, to feel their presence. And this is what it's all about: Some places had certain qualities that promoted a certain numinous ambience which could then be ascribed to the presence of a deity.