It's definitely possible, and many Classical scholars would say probable. The popular reputation of the famously mad, bad and dangerous Emperors of the early Empire (Nero, Claudius and Caligula) is largely down to their principal biographer, Suetonius. He weaves a purposefully racy yarn and is the guy from whom we get rumours about Caligula and his horse or Nero playing the fiddle while Rome burned. Suetonius wrote many emperors after these ones were dead and gone and he was a Senator, meaning he comes with a view from a very different time and a specific class (one which often clashed with emperors' autocratic tendencies).

There are certainly other sources for these emperors' lives, but they don't have the same kind of compelling (and pretty entertaining) biographic approach. Some of Suetonius's near-contemporaries and social equals (Tacitus and Dio) also paint a pretty negative picture in general, but not entirely and that doesn't really say much for how they were viewed by those they actually ruled.

To take Nero as an example, Miriam Griffin (Nero: The End of a Dynasty) and Edward Champlin (Nero) both provide really good analysis of a wide range of contemporary and later evidence that suggests that although much of his behaviour might later be seen as eccentric, it wasn't necessarily so extreme at the time. It's also, of course, not the whole story, and there was plenty of other rational, normal governing stuff that Nero did. That he was bestowed with divine honours after his death, that these were maintained and observed by at least some emperors in later dynasties suggests there was some popular goodwill towards his memory. His reputation, like those of many other emperors, ebbed and flowed with the needs and politics of the time, just like we judge them now against our expectations of what rulers are/should be like.

A terminological quibble, since I'd never heard "damnatio memoriae" before and had to look it up: Doesn't it refer to attempts to erase someone from history, not sully their reputation?