Children are capable, of course, of literary belief, when the story-maker's art is good enough to produce it. That state of mind has been called “willing suspension of disbelief.” But this does not seem to me a good description of what happens. What really happens is that the storymaker proves a successful “sub-creator.” He makes a Secondary World which your mind can enter. Inside it, what he relates is “true”: it accords with the laws of that world. You therefore believe it, while you are, as it were, inside. The moment disbelief arises, the spell is broken; the magic, or rather art, has failed. You are then out in the Primary World again, looking at the little abortive Secondary World from outside. If you are obliged, by kindliness or circumstance, to stay, then disbelief must be suspended (or stifled), otherwise listening and looking would become intolerable. But this suspension of disbelief is a substitute for the genuine thing, a subterfuge we use when condescending to games or make-believe, or when trying (more or less willingly) to find what virtue we can in the work of an art that has for us failed.

• J. R. R. Tolkien, "On Fairy Stories" (1947)

In his famous essay, Tolkien gives what would become a highly influential definition of ways to divide fantasy, between those stories which have a link (however tenuous) to the "real" (or "primary") world and those which are set in fantastic realms completely separate from the real world (what Tolkien called a secondary world). Tolkien made the distinction, but it was not by any means a rule or mode of fantasy fiction, something I touched on briefly in What was fantasy literature like before Tolkien, and how does it compare to Tolkien's works?

A great deal of fantasy fiction in the past took place or included fantastic settings, but the style of the fiction usually came from familiar types of literature. Gulliver's Travels (1726) for example is a fantastic satire in the form of a travelogue, and some of the claims in it for the lands and peoples discovered are no more outrageous than in The Travels of Sir John Mandeville (c. 1357), the Romance of Alexander, the 1,001 Nights and A Night, or the Histories of Herodotus. The pretense, for Swift in his fictional Gulliver, is that these are hypothetical lands that one could go and visit - even though he and his audience were aware they were a fabrication, the whole narrative partook of the nature of a fanciful account of foreign lands.

That being said, you do sometimes have examples of early fantasy fiction where the world or setting described is disconnected from the world that we know. As one researcher put it:

Maps of imaginary lands have been with us a long time. Essentially they are of two kinds: geographical speculation and literary fabrication. While the maps which most concern us in this book spring from literary fabrication, no category really holds up eternally.

• J. B. Post, "On Charting Unreal Realms," An Atlast of Fantasy

You've mentioned Thomas More's Utopia (1516), but we might also add John Bunyan's Pilgrim's Progress (1678) with its allegorical world, the fictional world of Allestone created by Thomas Williams Malkin and published in A Father's Memoirs of His Child (1806), L. Frank Baum's Oz (1900), various dreamlands including those of Lord Dunsany and H. P. Lovecraft, the fantastic planets of Edgar Rice Burroughs and E. R. Eddison, etc. etc.

J. R. R. Tolkien, it's interesting to note, did not start out with the conception of a completely separate imaginary world; Christopher Tolkien in The History of the Lord of the Rings notes how his father's conception of Middle Earth grew and changed over time, and there were edits made in some of the later publications to make it more distinctly not a part of "real" Earth history.

This is a very English literature-heavy answer; cases in non-English language literature are going to vary, although in cases like Journey to the West and 1,001 Nights you have, again, the adaptation of a known travel narrative which hits that frontier where the exotic-but-known becomes the unknown-and-fantastic. You see a lot of that kind of thing in science fiction as well...only applied to different planets! The same basic narrative works because people accept, as a rule, the geographical limitations of their knowledge.

Also as a rule, the point of these different imaginary worlds can be quite different. Swift was going for satire, Bunyan for allegory, Tolkien initially set out to create an English myth, Robert E. Howard with the "Hyborian Age" was creating an imaginary past for his sword & sorcery stories to take place in, H. P. Lovecraft and Lord Dunsany in their dreamlands were creating places of deliberate fantasy and beauty where the aesthetics were not limited by dull reality, etc. etc.

It's hard to say if there was a breaking point, exactly. The runaway success of Tolkien's The Lord of the Rings in the 1960s definitely spawned a large number of pasticheurs and followers, not the least of which was Gary Gygax with Dungeons & Dragons, but Gygax was also drawing on the work of writers like Fritz Leiber and his world of Lankhmar, C. L. Moore and the lands of Joiry, Howard's Hyborian Age, etc. Definitely, as fantasy really took off, Tolkien's essay and the distinctions he made - as well as Lloyd Alexander's definition in High Fantasy and Heroic Romance - have strongly shaped the fantasy worldbuilding scene. They were not alone in creating it, however.