This is an interesting question, if only because it really reflects the (western, mainly American) public's really skewed view on Christian interpretation of the Bible in the modern era. While I cannot comment on the other Abrahamic faiths' interpretations of the Genesis story, I am familiar with Biblical hermeneutics among the various Christian denominations. To be concise, yes, there was indeed a time when the Bible was not subject to a strictly literal interpretation; in fact, for much of Christian history, the Book of Genesis was explicitly not literally interpreted, with the rising trend of Biblical literalism only really picking up in more modern times. Now, for the sake of this argument, I'd like to clarify that when I say "Biblical literalism", I mean the full out belief that Biblical descriptions are not only the Word of God, but that they are also physically and literally true, as in the world was created in six chronological days and that the genealogies that listed Noah and his descendants as living for several centuries were grounded in physical reality, and so forth (distinction needs to be made as some denominations refer to themselves as "biblical literalists" in that they believe in the inerrancy of the Bible but not necessarily that the events physically transpired).

For much of the faith's history, Christian theology has taught that many stories in the Bible must be understood as vehicles for delivering God's divine message rather than as a literal account of the physical realities of the past. St. Augustine of Hippo was a prominent Christian theologian from the 4th century who wrote a 12-volume work (De Genesi Ad Litteram) explaining how it was necessary to use logical thinking when examining the Book of Genesis and to understand that its purpose was to explain the omnipotent power of God and His actions and purpose when creating Heaven and Earth rather than providing a literal description (though in the interests of full disclosure, he did advocate a literal interpretation of Adam and Eve). As the Christian movement became organized through a series of Ecumenical Councils, the Biblical Canon was decided and a common theological framework was established (though there were often many disagreements, leading to the splitting of the non-Chalcedonian churches like the Nestorians and the Copts). While the individual Christian branches differed in their interpretation of certain theological principles (the nature(s) of Christ, filioque, etc.), most of the various denominations before the Early Modern Period were characterized by established hierarchies of learned clerics whose duty it was to explain the Biblical passages to the masses. Most people at the time were illiterate, and the ability to read was a luxury often restricted to the highest nobility and the clergy. As such, the great masses of the people relied on their spiritual leaders to interpret the Bible's messages for them.

This all changed, however with the Protestant Reformation. While Luther's rejection of the rigidity of ecclesiastical hierarchy as necessary for development of faith was certainly an important factor with the rise of Biblical literalism, the PR simply came at an extremely convenient time alongside (or some would say, because of) the development of the printing press, the dawn of the Scientific Revolution and the rise of empirical modes of thinking. The ease of production for texts meant that a much wider audience was now able to have access to their own copies of the Bible (though literacy was still rather poor). Couple that with the teachings of Luther and Calvin which stressed one's own personal interpretation of the Bible without the "meddling" of (what they considered) meddling, papist clergy, a great many people were thus motivated to read the Bible and interpret it of their own accord.

As time passed and the various Protestant churches evolved, the Bible, not the church, became the final authority of all theological disputes, with adherents deferring to their own interpretation of the Bible instead of those of past theologians, like St. Augustine. The occurrence of the Scientific Revolution was significant in this regard in that it placed an incredible emphasis on empirical thinking, that is, understanding ideas through their physical, observable elements instead of just theory. As such, many Bible-based religious communities focused on the literal interpretation of the Bible instead of the numerous other hermeneutic approaches (such as the historical-grammatical approach that takes a more holistic view of the context of a passage, its audience, the subtleties of language, etc.). As time drew on, certain religious denominations came to view Biblical literalism as a defining part of their faith identity, the very attribute that sets them apart and makes them "special" among other denominations, thus leading to an increased unwillingness to depart from such a view.

TL;DR Genesis really wasn't taken literally (in the sense we know it now) until post-Protestant Reformation, where the availability of books and an obsession with physical observation encouraged people to focus on the literal aspects of the Bible.

For additional readings, check out Diderot's Encyclopedie, which is one of the earlier works addressing Biblical literalism in the modern sense as well as Laurence Woods's Theology as History and Hermeneutics and Ramm's Protestant Biblical Interpretation.