Good question. In fact, the question is so good it has been asked and answered several times before and is now prominently featured in the Frequently Asked Questions section. There's an entire subclass devoted to it.

• What caused the fall of the Roman Empire?
• How sudden was the fall of Rome?
• Did it actually fall?

I see that the phrasing and direct aims of those questions slightly differ from yours, but together I think they've got it covered.