I can answer for Hellenistic Greece and the Roman empire.

The major thing at issue here is, "What does it mean to be a god in any particular culture?" Modern Westerners tend to have a lot of trouble with things like ruler cult because our cultural idea of a god completely precludes any humanity. We've essentially adopted, though Christianity, a neo-Platonic idea of deity as originating outside of the universe and therefore completely alien to it. (This is why orthodox Christian theology can get so messy--it's trying to philosophize something that the philosophical system it's based on fundamentally rejects.) When we think about polytheistic systems, this causes us to emphasize the genetic difference between gods and humans, and also contributes to a difficulty seeing those gods as anything more than "mythological."

Polytheistic societies often don't have this problem. In the traditional, nonphilosophical Greek and Roman traditions, for example, the universe is entire. There's nothing besides the universe, which is roughly the size of the earth plus some room for (much smaller) astral bodies, therefore nothing outside of it, therefore gods are part of it, therefore they're part of nature. They can do a lot of cool things that humans normally can't do, but humans can do a lot of cool things that, say, cattle can't normally do.

Also remember that there's no such thing as genetics in pre-modern societies. There are ideas about "nature," so you look like your parents and your friends look like their parents and your dog looks like its parents, etc., but it's a much soupier concept than genetics, with different answers in different cultures. But that often leaves room for a lot of squishiness between natures.

So, gods. SRF Price tackled this for the Roman imperial cults ("cult" = methods and means of worship in this context, not groups of people like we use it today), and found that, yes, people did really believe, with two caveats: first, like I've already said, the definition of "god" is different between the question you're asking and the answer the Greeks and Romans are giving; second, while it's too much to say that there's no such thing as belief in this situation, it is true that the Greco-Roman systems don't care about belief in the way Christianity does. What matters is that you perform the correct actions--sacrifices, festivals, taboos, etc. When we look at trials against Christians, they're not asking them to stop or start believing anything. They're asking them to DO things: sacrifice to the emperor and/or the gods.

For the Greeks and Romans, the definition of deity had a similar emphasis on function. Gods are gods because of what they do: they protect us from military conquest, provide food and other resources, heal sickness, etc. They maintain order in the universe, which is, again, much smaller than our modern understanding. Here comes Augustus, who ended decades of war, got trade moving, made food more available, and even at times provided grain at his own expense. Now, there is definitely a ton of propaganda surrounding Augustus and his divinity, but it's based on a tradition going back in Greek society at least a few hundred years, which is that when a king or general provides major benefits to your city, you reward him with godhood (notice here that the king isn't forcing anyone to do anything--ruler cult in Greco-Roman societies was usually granted and often unofficial and spontaneous, so that the sources tend to heavily look down on the rulers who demanded it). I'm on mobile right now so I can't quote it exactly, but if you Google Augustus's birthday announcement from the province of Asia you'll see a really good example of this. When you look at the language it uses, remember that it was written decades before Jesus was born.

You might have caught some inconsistency there: is someone a god because of what they did, or because we worship them as a god? Basically, yes. In nonphilosophical, non-Christian Greco-Roman culture, beliefs aren't codified. Actions create beliefs and vice versa. The problem of nature also creeps back in here, and it has cultural boundaries. Romans, at least officially, liked to emphasize that the emperors were divinized, not divine by birth; Greeks didn't worry about this so much. In Latin, Augustus is "son of the divinized one," meaning Julius Caesar. The Greeks translated this "son of god," even though the Greek language is perfectly capable of expressing a passive causative participle. There also developed birth narratives: Augustus's mother fell asleep in the temple of Apollo and was impregnated by the god. But it's important to say that these usually seem to pop up a while after the cults, suggesting that having a divine birth isn't necessary for divinization but sometimes might make people more comfortable.

There isn't a lot of emphasis on afterlife--most Hellenistic rulers and Roman emperors did not continue to receive cult for very long after their deaths (hero cults, which were dedicated to deceased people, were a slightly different thing, sometimes derived from the cult of a living ruler after his death, but it wasn't a necessary development and they didn't always begin that way). But there also, just in general, wasn't a lot of emphasis on afterlife at all in this period. There were various beliefs about it, but it wasn't a major cultural force like it became in Christian Europe. So the answer to your question for a culture with a more robust afterlife tradition, for example, dynastic Egypt, might be very different.