There are two different issues here. One is authenticity, the other reliability.

Determining the authenticity of a piece is important, but we have a number of available tools. If it's a more recent document we can analyze the paper (or parchment), ink, handwriting, and grammar/vocabulary to determine whether it appears to have been written when it claims to be, and whether it matches other documents (in handwriting, writing style, vocabulary, etc) from the same author (if any such survive). This is more challenging with older sources, which have often been copied several times before we get them (like, for example, the writings of Plato), but we can use similar tools to analyze the style, grammar, vocabulary, etc to determine if the text is a copy of an original, or a forgery. There have been a number of famous texts (like the donation of constantine) that were identified as later fakes through these methods. All told, we can usually tell if a text is authentic (though some texts continue to be debated).

The question of reliability is much more challenging, and gets into questions of interpretation that are fundamental to the work of historians.

If by reliable we mean 'trustworthy sources of facts that we can count on being true,' we have to throw out most of our sources. Even the most accurate eyewitness is limited by his or her perspective, and faithfully reported 'facts' are always subject to both the interpretation of the person who reports them and the person who reads and interprets them. If our goal were to try to find only the things we could trust for certain in ancient documents, history would be a sad and small discipline.

Instead, when historians read texts, we generally don't try to separate 'fact' from 'fiction' from 'bias.' Instead, we try to figure out what perspective someone was coming from, why they reported what they did, how their perspective, biases, and unique points of view shaped what they reported, and (most importantly) what they were trying to accomplish when they wrote the document we're reading. Bearing all this in mind, and reading these texts against other documents from the period by the same or different authors, we can begin to see the specific choices the author made when s/he wrote the document the way s/he did. We can start to see which facts the author chose to include, and which s/he ignored. Which facts the author appears to be spinning a certain way, and which the author thinks are really important for answering a certain kind of question. This shows us what the author values, how the author worked with his or her evidence, and allows us to start to form an idea of what the actual events the author was responding to may have looked like.

Depending on the type of document we're reading (narrative history, literature, census data, legal code, etc), there are different specific sets of tools that historians generally agree are more or less effective. But in general, all historians are always analyzing sources to see what their goals, objectives, and unique perspectives are, rather than simply trying to separate 'fact' from inaccuracy. Everything is interpreted when it's written down, so we have to re-interpret it by reconstructing as much of the context in which its author was working as we can. It's very challenging, and one of the many reasons why our understanding of the past is always growing and changing as we learn new ways of understanding what people in the past were trying to do when they wrote the things that we use to learn about them.

This is a bit of a difficult question, but I'll try my best to answer it with what I know. I helped my resident professor write a series of novels. Mostly these weren't "published" but were put in academic journals. What historians look for in sources is, primarily, consistency. Cross-checking sources is a must....but at the same time how many sources needed or if the basis of the statement is valid comes down to the Historian's personal views. I'm not saying every bit of history is 100% biased propaganda, but a lot of it is the historians Educated opinion on the matter. For example, my resident professor wrote a short paper (for him that's 50 or so pages) about the history of armored warfare. He had to go through hundreds and hundreds of sources to verify certain numbers. Some sources are completely unreliable in one sense but not in another. An example of this is a Period Piece or a work (of fiction or not) that can't tell the truth through it's own words, but gives insight on the period in question. A really good example of this is All Quiet on the Western Front. It's based on reality, set in reality, but it's a novel, it's fiction....but it still gives a great amount of information on how the Great War affected the common soldier. I wouldn't use it as an example of real history, there's no Paul Bäumer out there who experienced that story personally. But, All Quiet on the Western Front is a work of fiction that gives insight on how veterans felt after the Great War, Lost, Broken, and generally traumatized. A counter to this piece would be Storm of Steel which is a personal account of WWI by Ernst Jünger. It's dry reading for non-military history nerds, but it provides rather quick and to the point descriptions of what happened to Ernst. I hope I can help.