First of all, evidence is a tall order. History is politics, and recent history is dangerous indeed to write in a tumultuous empire such as Rome. No historian (or, at that, poet) wrote outside the shadow of the Emperor's influence. Claudius himself was a historian and wisely (mostly) kept to history far enough in the past to have no current political consequences. Others weren't so careful and got exiled for it.
It was in every emperor's interest to make the previous emperor look either good or bad. If he was the (typically adoptive) son and heir of the previous emperor, he took his father's name, likely deified the deceased ruler, and was a son of a God. He was a legitimate continuation of a dynasty, and writers of the time would find it prudent to speak well of the Emperor's father. If, on the other hand, the new Emperor had found his elevation through military might or his mother's political machinations, it was in his interest to make the previous ruler look like a real sonofabitch. In other words, if the new Caesar might not be seen as legitimate, people must believe there was a real good reason the last guy was poisoned/stabbed.
In any account, you have to consider the political situation when it was written, how much time had passed since the event being considered, and use your own good judgment.
For example: Messalina, one wife of Claudius, was said to have entered into a competition with a prostitute to see how many men they could bed in a night. One can hardly imagine a story more offensive to Roman moral sensibilities. And it seems just pretty damned unlikely, especially considering the people who wrote about it had a personal interest in damning her name.
Damning her name: Damnatio Memoriae. That's what the senate ordered. It meant a person's name would be scraped off any stone, every public building, and their head would be lopped of any statue, maybe to be replaced with the head of a God. Usually this is understood as erasure from history, but that's not realistic. I believe damnatio memoriae also meant including their names in history, but saying really bad things about them.
Elagabalus was said in the notoriously unreliable Augustan History to have prostituted himself and bragged that he got more lovers than the professional prostitutes. Sound familiar? Oh, and he was also a member of the Damnatio Memoriae club. Another calumny, very possibly. Augustan History was written under Constantine, by the way, and Elagabalus was your Pagan's Pagan--he elevated his foreign sun god over Jupiter! Offensive to Christians and traditional Romans both.
Caligula made his horse a senator. This one's more plausible. Sounds crazy, but not so offensive to Roman sensibilities that it strikes one immediately as calumny. In all likelihood, he was mocking the senate, as there's always struggle between Emperor and Senate. "Hey, you guys suck. My horse could do your job! Gallyup, Incitatus!"
Commodus was a bad emperor. Everybody said so. They were probably right... but you can't necessarily believe the most out-there stories about him, because in the end he was declared public enemy by the senate. They had to curse his name or his assassination wasn't legitimate, you see? In his case, he had to be cursed all the more strongly because he couldn't have been MORE legitimate--he was the rare natural son and heir of the previous emperor!
In any case, an Emperor valued his name's memory more highly than his life... and a victorious rival ruler took pleasure in stamping it out.