Caesar's career was a major factor, perhaps the deciding factor, in the fall of Republican government in Rome, and the subequent rise of his nephew Octavius/Augustus established the Principate/Imperial government, but all of this happened in the 1st century BCE and 1st century CE. The dissolution of the Western Empire occurred in the 5th century. These are two vastly different historical periods. In between, the Empire reached it's greatest extent of territorial dominance and internal stability, particularly in the 2nd century under the Antonine dynasty. The crisis of the 3rd century (235–284) saw 22 Emperors in a period of 50 years, few of whom died natural deaths, but the Imperial system didn't collapse. It was somewhat re-invented by Diocletian in what is known as the Dominate (as opposed the previous Principate) to continue for almost another 2 centuries.

The fall of the Western Empire was the result of many factors, both internal and external, too many to list and all a matter of debate as to their ultimate importance, but "a series of astoundingly terrible emperors" doesn't correspond to any theory of causation I'm aware of. Certainly the problem of succession caused frequent instability and occasional warfare, certainly any monarchical/autocratic system faces the problem of being stuck with the occasional bad Emperor, but the Roman Imperial system proved remarkably robust considering all the blows it received.

Peter Heather's "The Fall Of The Roman Empire" is a good accessable introduction to the question of decline and fall.

Edit: I now can't get the idea of Lemony Snicket's A Series Of Astoundingly Terrible Emperors out of my head.