It was a long, slow fall, more with a fizzle than with a bang.
In many ways, most of the Empire's history was a steady process of decentralization. Note that 'decentralization' is not a 'fall' or even necessarily a 'weakness' (see Peter Wilson's "Heart of Europe" for an exhaustive discussion on this). But it does mean the component parts of the Empire (the Imperial Estates) became more independent, and relied less and less on the Empire's central structures. Note that when the Empire fell in 1806, having a single strong leader with near-absolute power was considered a plus, so the Empire was looked down upon and that perception is propagated to this day. But nowadays, we view such a leader as a dictator: we value decentralized power. Historians are slowly coming around to refurbish the image of the Empire. That's why some now look to the Empire as a model for the modern European Union, but Wilson disagrees with this view as well.
The canonical date for decentralization people often cite is 1648, the Peace of Westphalia ending the Thirty Years War. But it should be noted that over a hundred years earlier, Martin Luther initiated the Protestant Reformation within the bounds of the HRE. The staunchest supporter of Roman Catholicism at the time was Emperor Charles V (his sacking of Rome excluded). Charles V is often cited as the most powerful man of early modern Europe, his personal possessions including Spain and its New World holdings, southern Italy, Milan, Austria, the Low Countries, as well as significant parts of modern France, Germany, Slovenia, and more. Yet even this powerful man could not put down Luther, such was the control the individual Imperial Princes (notably the Elector of Saxony in this case) had over their own territories. To be sure, if Charles V wasn't bogged down in so many other wars (notably with the Ottomans) he probably would have been able to stop the Reformation (though that's just my personal opinion, actual historians differ).
The end of the Thirty Years War in 1648 really just codified this independence further, especially in regards to religious matters. Note that while the name "Holy Roman Empire" was rarely used throughout its history, the notion that it had a "Holy" mission was nonetheless ever-present. So its abrogation of its Catholic mandate certainly weakened the perceived power of the Emperor.
With Prussia's rise, which the Habsburgs were unable to stop, there became a second great power within the Empire. (Both Prussia and Austria had significant holdings outside the empire that contributed to this power.) Back when the Habsburgs were so much more powerful than any individual state in the Empire, the Emperor could project power beyond the clearly codified legal powers he already had. But when the Prussians could challenge the Habsburgs, the legal powers of the Emperor were minor and this reduced the Empire's standing further. This same situation holds for most medieval states: the kingdom was really only as powerful as the king. The HRE was a medieval state that stayed alive into modernity, but could not adapt to its contemporary fiscal-military competitors. By the time of the War of Austrian Succession, we generally speak of Prussians and Austrians with little regard to the powers the Habsburg Austrians had within the Empire.
Enter Napoleon. By 1804 it became clear that Napoleon might well be a permanent fixture in the lands of the Empire. Foreseeing the end, Holy Roman Emperor Francis II decided to take a second imperial title, becoming Emperor Francis I of Austria. After the battle of Austerlitz (1805) completely humiliated the Austrians, Napoleon created the Confederation of the Rhine (July, 1806), separating most of the smaller imperial Estates from the Empire. In August 1806, Francis abdicated his Holy Roman imperial title, dissolving the Empire, largely confirming reality but also apparently out of fear Napoleon himself my claim the title. After Napoleon's fall in 1815, the Empire was not reconstituted, instead the German Confederation was created, it too suffering from Austro-Prussian rivalries.