There is a question within your question that is central and all important as we conclude that there is no way of knowing.

First we must ask, what are the gods? While that answer may seem obvious to the modern mind, the obvious answer is not necessarily the one that is appropriate for the ancient and/or prehistoric world. We think of "God" as the supreme, all-knowing, all-powerful entity, the Alpha and the Omega, etc. But as we begin looking at the "gods" (plural), it is fairly obvious that these entities represented checks on one another in any kind of pantheon and cosmology. They could not be, consequently, all-knowing and all-powerful since they shared power and had unique perspectives. It is, consequently, probably improper to use the term "gods" to describe them - at least without a great deal of caveats and understanding of the limitation of the term in this respect. We use that term because it is the one we have, but that doesn't make it entirely appropriate.

I prefer to think of them as powerful supernatural beings - some with more and others with less power. So your question is either where did the idea of "God" as we know it come from and originate? I'll leave that to the historians of theology, but I don't think that's really what you're asking about since you used the plural - gods.

So then the question is when did people begin believing that the world was populated by exceedingly powerful supernatural beings? The answer to that question is that we just don't know. Archaeology from prehistoric times hint at a level of spirituality, but we are projecting a lot - probably too much - when we attempt to interpret ritual burials and/or something like rock art. These things are compelling, but they leave us with more guesses than answers.

We can say that nearly every culture emerged into the historical record with a pantheon of powerful supernatural beings.(Askanthropology regularly debates whether that should be "every" or "nearly every", so I will be conservative here.) The implication is that we can expect modern homo sapiens to have shared in this cultural element for at least several tens of thousands of years. When (and/or where) this began is anyone's guess, and anyone who maintains he or she has the answer is passing off a guess for certainty.