There are a few things that have to be cleared up here.

• Looking for actual belief is a very Abrahamitic way to approach the question. In most historical belief systems, ritual was far more important than belief.

• By separating folktales from belief you are also approaching the question from a presentist, "scientific" perspective. For most of history there was no clear division between the two; folktales could be widely believed, and dogma did not necessarily exist to any great extent in every time and place

• Works like the poems of Hesiod, which I imagine you might be having in mind, were never sacred scripture. They were stories some people believed, others didn't, and others still didn't care about.

So how can we know what was important? Well, one thing to consider is that sacrifice and ritual took time, resources and effort. They were done because they were for one reason or another thought important. Another way looking more specifically at narratives is to consider their preservation. If oral tradition preserves the archaic language of some holy tradition over a milennium or more, while otherwise evolving, (as with e.g. the Gathas of Zarathustra) that suggests it has a particular reverence for the words of those verses that goes beyond what you'd expect for "mere" folktales. The use of a separate liturgical language instead of vernacular is a similar indication.

Another thing we can look at is what authorities invoked to shore up their legitimacy. If Hammurabi emphasizes the validity of his decrees by depicting himself as receiving a scepter of authority from the god Shamash, that suggests people attached importance to Shamash's divine authority.

Exactly how the divine was perceived varied between cultures. Many cultures strongly identified natural phenomena, like weather, with the divine. Some, like my preferred study objects the Indo-Iranians, even had a penchant for deifying incredibly abstract concepts - my favourite example is Vishnu, who embodies the concept of pervasiveness. To these people, we can tell from the ambiguous language of their hymns, there was no clear demarcation between e.g. Agni as the name of a deity, and agni as a noun meaning fire.

I have tried to answer this question as generally as possible. If there are more specific mythos or aspects you are interested in, let me know and I can elaborate or provide reading!

/u/Icnielsen does a great job here. There is another way to tackle your question: the term folktale is a tripwire, because folklorists - following the lead of the folk - use it to designate an elaborate fictional story, the oral novels of the folk. Pre-industrial Europeans use a variety of terms to separate legends from folktales: legends to be believed; folktales told for entertainment. Boundaries could blur as narratives of both varieties slipped into the other category.

Given that this separation of types of oral narratives are found internationally (though not not necessarily universally), it is likely that ancient people told narratives not to be believed. I have often wondered about the story of Jason and the golden fleece. Folklorists collected this widespread narrative in more recent times and classifying it as the folktale ATU 313 (Aarne-Thompson-Uther). Did the ancient Greeks anticipate later Europeans and tell the story of Jason merely as entertainment of a fantastic, "once upon a time" past? I'm not sure. Perhaps it slipped back and forth between genres as folklorists observed in more recent times. But disbelieving the Jason story is not evidence that ancient Greeks did not believe in the supernatural beings that manifest in the story. European folktales include a variety of supernatural beings: they describe ghosts and fairies in stories that the folk felt were fiction, but they also told legends about fairies and ghosts that people believed.