To answer this, I'm going to first start with an example and work through why the historical account is contested. One of the most infamous cases of this is probably Caligula, the "depraved" Roman Emperor. In pop culture (e.g. things like the series I, Claudius) he's portrayed as a truly despicable human being. He murdered people for no reason, openly slept with married women, wasted money, claimed he was divine, and slept with his sisters (he eats his sister's fetus in I, Claudius...there are no historical accounts of that happening).

These stories are from a variety of stories. The main two who were contemporaries of Caligula were Philo of Alexandria (in his work Flaccus where he discusses Caligula's treatment of Jews...Caligula claimed to be a god and they refused to worship him) and Seneca the Younger (whose analysis of anger in De Ira is considered a thinly-veiled critique of Caligula...though he does not mention Caligula by name). They paint the picture of someone unstable and crazy (considering himself a god and punishing the Jews for not worshiping him) and unstable and full of rage (challenging the gods to fight and lashing out at all those around him).

The other two sources, and all of the "juicy" stuff that people remember come from Suetonius (in his work De Vita Caesarum - The Twelve Caesars) and Cassius Dio (in Historia Romana). These are where the accusations of incest, wasteful spending, and the desire to appoint his horse a consul of Rome. The important thing to note here is that Suetonius was born almost 30 years after Caligula died and Cassius Dio was born over 100 years after Caligula's death.

So, that leads us to our first clue: the truly "juicy" stories don't come from his contemporaries but from people who were born decades after Caligula's death. Suetonius' work is, essentially, a massive piece of hearsay. Interesting for sure, but ultimately problematic when compared with previous writings on Caligula. It's possible they had access to sources that have since been lost but this isn't the only reason to be skeptical.

The other major reason that these stories are suspect is that Caligula was...not well liked by some powerful people. The Senate in particular hated him. In the end, he was assassinated by members of his own Praetorian guard. These same people would have a vested interest in painting Caligula in the worst possible light and spread scandalous stories about him to make the assassination seem more "justified".

So, to sum up, there are a number of things historians look for. Consistency in accounts are particularly important. It's likely that Caligula did have a temper as both Philo and Seneca mention it. It's not likely that all of the salacious details from after his death are accurate. Contemporary accounts are also incredibly important as it's less likely the stories will be distorted over time (e.g. perhaps Caligula was not serious in his threat to make his horse a consul, it was a way to call his opponents in Rome incompetent, but that's not how it was told to Suetonius). Looking at the motivation of those writing/relaying the history is also quite important as a highly biased source is not as trustworthy.

edit: noticed a couple typos

edit2: I should also note, history is not as...clean as most schools before higher education tend to show. In many cases there is no one "truth", even for more recent events than Ancient Rome. This is where lively debate comes into play. Historians will get into - sometimes quite heated - debates about what truly happened as you will frequently have sources that either contradict each other or make outlandish claims. Historians try to create a plausible hypothesis for events based on their evidence and present their findings to the academic community.