For one thing, other evidence. Coins, references by other contemporaries, texts chiseled into stone or worked onto metals, along with other archeological evidence. For example, for a long time Troy's existence was uncertain. Many considered it to be a fictional city, much like Plato's Atlantis. Then figures such as Heinrich Schliemann started using the epic poem as if it were a work of mostly non-fiction, in an attempt to find Troy. He did. Since then we have a much more nuanced vision of the poem itself, noticing that some sections of the text are clearly from an early period of Hellenic history while others are anachronisms that point to later additions. A great book about this that I loved reading was The Bull of Minos, by Leonard Cottrell. We also keep finding coins with the names of rulers we thought were myths and the ruins of elegant ships whose luxury we had thought an exaggeration, among other things.

Another key piece is comparing stories from contemporaries, or at least almost contemporaries. Often times comparing texts will bring up telling contradictions or omissions, which together with the next point help us figure out the motivations and reliability of an author. Sometimes, especially in the more chaotic moments of history, we may have only one source relating X or Y event, or characterizing ruler A or B. That brings us to the last point.

Finally, critical methodology. As a historian, when you analyze the text you have to take into account a) who the author was and what interests he represented/sympathized with, b) when the author was writing (e.g. was he writing about something he could not have been an eyewitness to?), c) does he cite sources and do we have access to said sources (what are the reliability and motives of said sources, how critically does he/she approach them), d) do we have reliable confirmation of some or most of the text through other kinds of evidence (e.g. archeology, contemporary authors, literature/art from the period, etc.). These are the kinds of things you have to take into account. A major pitfall many popular historians fall into is uncritically accepting sources, such as the scandalous though entertaining anecdotes of Suetonius, without critically analyzing the source and its reliability using critical methodology. A more contemporary example would be Cuban history after 1959. Many sources claimed privileged knowledge about X or Y moment in the Revolution. But you have to take into account what their personal stake is in what they're writing; cui bono (who benefits from their difficult to confirm but damaging statements?); how much proof did or can they provide for anything they're saying? This is true of both sides: How do defenders of the Revolution evidence their explanations for embarrassing moments? What do they choose to omit? Are they mostly free from political/economic/social pressure to tow the line on certain issues? What evidence can or do they provide for their attempts to explain away this or that issue.

These are some of the basic pillars that help prevent major mistakes, such as attributing seriousness to authors who may be 'having fun', as you say. Sometimes, imaginably, some may get through and we may take their commentary too seriously. We may misinterpret archeological evidence, favor X author above Y because of some kind of bias, etc. It can conceivably happen. However constant debate among historians in academic journals, books, and even news articles, help provide venues for new, critical interpretations of what we think we know about the past. Ideally, no fanfiction should survive the gamut of critical analyses for too long.

Funny, I just finished a presentation today on Hagiography and the problems of our modern representation of it as "fiction."

A lot of the problem stems with differing conceptions between the modern understanding of "truth" which is based on positivist / enlightenment-era understanding of the sensory world, and ancient understanding of "truth" in its platonic form, that of an "ultimate" but "hidden" truth.

There are still elements of this philosophical conflict in everyday discourse you might see on Facebook, or Reddit. Whenever you see someone yell "WAKE UP SHEEPLE!!" (or its equivalent) over some political problem, what they're trying to do is claim a higher hidden truth over the "material" facts on the ground. Frequently the "truth" the person is trying to wake people up to is a hidden system underlying the sensory details that we might all agree on, a hidden "truth."

The Platonic understanding of a hidden truth was everywhere in Antiquity, be it Christian or "Pagan" (usual caveats applied to this word). So when historical texts are written with an eye for truth, they mean to portray "higher truths" about a situation, a philosophy, or the character of that individual. And since the goal is these higher truths, what matters less is the actual factual events on the ground (which lets be honest, would most likely be near-impossible to capture, especially back then), so long as events were "within plausibility", they were taken as "good enough."

Because in this post-linguistic turn era, we have to treat all history as "text", we can't even be sure that quotes said by people were actually said by them. We in fact, have to (and should) suspend some of our belief into what constitutes "actual." But once again, we should also keep in mind that this flows from a particular 19th century philosophical perspective, which laid precedence on a particular kind of "actuality."

Even now, in an era with supposedly "objective" recording devices, like cameras, or microphones, we still argue about the "reality" of the events depicted. Even now, if you were to argue with a person who called you a "sheeple", you're still disagreeing over the fundamentals of what constitutes "real" and "truth."

So, to answer your question, how can we tell if a historic text isn't just fiction? The long-con answer is: We Can't.

What does that mean for the study of history as its practiced now? If we focus less on the "actual" and more on the "way", we can still reconstruct a lot of the past. Which means, it matters less whether or not Julius Caesar ACTUALLY said "veni, vidi, vici" or whether he ACTUALLY crossed the Rubicon river, than that people thought he did, and that the way they thought of it had implications on people's lives throughout that time frame.

Something to think about. We can't even tell if news articles we're writing about NOW are "true" in an absolute philosophical, with our ability to physically go there, and interview witnesses, or even witness it ourselves. "Truthiness" is as powerful now as it ever was.

So, if that question can't be resolved with crystal clear 100% certainty, it's best to side step it, and look instead at what effect these "realistic fictions" have on people.

As a medievalist, I assume all my narrative sources (histories, Chronicles, etc) are fiction.

Early medieval authors cared about telling true stories, but they didn't have the same idea of what a true story was that we do. Modern historians are interested in details - names, dates, motivations, little things that fit together into our arguments. Early medieval historians, however, were much more interested in the big picture: the morals of the (hi)story, the lessons to be learned from the main players, the way the events described fit into the bigger picture of God's plan and other, similar events. Important parts of the story could be emphasized by changing the details to refer to other stories (for example, switching the order of events to make the lesson easier to understand, or stealing an event from a completely different text, rewriting it, and using it to show how your story tells the same spiritual lessons as the story you stole it from). Things like plagiarism, copying motifs and themes to show connections across time and space, and using stock tropes to tell your audience how to interpret characters' actions (ie, a king getting angry = a common way to tell the audience the angry king could not control himself or his kingdom, not necessarily an actual thing that haplened) - these were all as normal and honest to early medieval historians as footnotes are to us.

And miracles are proof that something really and important happened, so they have to be included whenever the author wants to make sure we're paying attention.

Reading these texts (say, Gregory of Tours' Ten Books) as records of fact would be profoundly naive, and most medievalists haven't done this since the 80s. But when we read them more like literature - as stories with important messages to share, rooted in fact but not describing it literally - they can still be incredibly useful. Just like fiction - which also tells important messages that resonate to its contemporary audoence - is useful for understanding the time and place from which it came.

So when I read historical texts writen by early medieval authors, I do think about them as stories rather than as history (that is, history in the modern sense). But stories are useful - it's fiction, yes, but not just fiction!

(And of course, I look at many other things besides medieval texts - archaeology, art, coins, environmental studies, geography, etc. these provide a context in which medieval stories can start to make sense.)

This is not really a new problem.... it takes place all the time. Please remember though that books are not just "things" that people pick up and read now and then. In ancient times books cost a lot of money to produce. John J. Phillips, in his article Book Prices and Roman Literacy, estimates that even cheap books cost two to three days wages written on the cheapest material in the 1st century. Then a book needed to be transcribed once the book got old to survive to the present. Each time it is transcribed it gains mistakes made by copyists who might get something wrong. Many books have been sadly lost to time. "Degrees" didn't really exist either as to be taught you needed a tutor which costs money. This means only a fraction of the population is literate and able to write a book even if they wanted to.

A great example of a problem you brought up can be found in the Travels of Marco Polo. A man writes his travel memoirs which many scholars continue to debate. Did Marco Polo really go to China and document it? If so what parts are accurate and which parts are hearsay? What did Marco Polo say just for the shock value and popularity vs. what he actually saw and did? Historians must analyze clues supported by other data to get a better picture of how a text fits in its historical setting. No historian just takes all texts as truth, but instead pieces it together and sees how it may or may not fit within a historical settings' context.

Fiction from a time period is still an extremely valuable historical source. Writers tend to write what they know, so while it might not contribute to the timeline of a certain period of history, in many cases it can offer extremely valuable insight into the time period in a more general sense. And example would be the pillow book from ancient japan which offers a fantastic view into the life of a courtier woman during the heian period.