Well, I'm by no means an academic, but overall it wasn't holy, as it wasn't led by the Pope or any other religious authority (and the Emperor was often arguing with the Pope), wasn't Roman as it was predominantly German (the full name of the state was something along the lines of 'The Holy Roman Empire of the German Nation'), and only directly controlled parts of Italy for a parts of its existence, and was not an empire as it was made up of hundreds of independent sovereign states, from as small as a city to as big as Austria or Bavaria, with an Emporer who was elected by certain member states and only posessed nominal central authority.

Edit because I'm too inept to use my phone keyboard correctly.

First, it's important to note that in the first phase of its history, the realm was referred to simply as Imperium Romanum, the Roman Empire. This was because the Franks actually controlled the city of Rome, beginning with the reign of Charlemagne. As a reward for saving the city from other Germanic tribes, the Pope conferred upon Charlemagne the title of Roman Emperor. This is where the "Holy" part of the name comes in - because no Western Emperor had ever been crowned by a clergyman. Charlemagne and the Papacy were forming a new precedent that is as clear a break with the past as the fact of a German becoming Emperor. By having the Pope crown him, this new Western Empire was founded with the Pope and the Catholic hierarchy as a far more important pillar of support.

Charlemagne was reluctantly recognised as Roman Emperor by the actual Emperors in Constantinople who had legitimate claim to the title. But as time wore on his successors lost the recognition of the eastern emperors. The term "Holy Roman Emperor" thus developed as a way to differentiate the Western Emperors, crowned by the Pope, from the Eastern ones, especially after the Great Schism of the 11th century, following which Eastern and Western Christians both saw each other as heretics. It began to show up in official documents in the 12th century.

When did the the Empire stop being "Holy"? We must recall the particular situation of the Papacy when Charlemagne was crowned in Rome. In the preceding two centuries, the See of St. Peter had been under the control of the Byzantine Emperors, who had maintained a tenuous grasp on Italy. During the "Byzantine Papacy", all appointments to the Papacy had to be ratified by Constantinople. When the Franks took over, the Catholic Church saw an opportunity to place itself above the secular rulers. Now, the popes made emperors instead of the other way around. However, the Germanic rulers did not always see things that way. Their power over Rome became ascendant in the reign of Emperor Otto I, who even dethroned a Pope during the period of the Pornocracy.

A reaction eventually sprung up around Pope Gregory VII in the late 11th century. This reached its climax with the Walk to Canossa, in which he forced Emperor Henry IV to walk barefoot and kneel for three days and nights in a blizzard in order to receive a revocation of his excommunication. But others will say that the Empire truly ceased to be "holy" when Charles V outright sacked Rome in the 16th century.

We should remember, however, that the Imperial title was always "Roman Emperor" and, later, "Roman Emperor-elect" - the word "Holy" was never included.

As mentioned earlier, the ancient line of Roman Emperors ruling from Constantinople soon stopped recognising the rulers of the West as equals, only legitimising the reigns of Holy Roman Emperors when it was diplomatically expedient. Nevertheless, the Western Emperors could claim some legitimacy for their title from the fact that they actually controlled Rome, which the Byzantines hadn't since the 8th century. However, Rome itself was under the title control of its bishops, and relations between the Emperor and the Pope could easily be mistaken for diplomacy between foreign powers. Incidents like the Walk to Canossa severely strained relations. Popes routinely raised up pretender Emperors and sought more control over the Church hierarchy in Imperial territory. To maintain their independence, the Popes supported the cities of Northern and Central Italy in their constant struggles against Imperial control. Eventually this pushed the limits of Imperial power further and further north. Charles V's Sack of Rome in the 16th century merely gives the lie to his "Roman" title - if Rome truly was his capital, why would he have to besiege and sack it?

After Charles, the Emperors were not even crowned in Rome any more. The political situation in Italy simply made it too dangerous for an emperor to make his way to the Eternal City. As a compromise, an emperor would be crowned as "Roman Emperor-elect". This can easily be construed as a farce.

What about being an Empire? Early Frankish custom was to divide a king's territories upon his death among all his sons. Instead of inheriting everything as became the norm later, the oldest son merely took precedence amongst his siblings. There are more complicated dynamics, such as the structure of Imperial Circles, which led to the Empire's fragmentations, but this division of inheritance laid the seeds for the outlandish growth in the number of states in the Empire, while also causing its outlying territories to break away completely. Charlemagne died ruling France, Germany, Italy, Illyria and even parts of Poland. By the time of the Thirty Years' War, this had shrunk to just Germany and parts of the Low Countries.

Furthermore, the emperors persistently failed in their quest to become absolute rulers. Old Germanic ideals of collegiality meant that emperors were heavily dependent on the Imperial Diet, a parliament composed of the hundreds of princes of the Empire. The Emperor himself had no direct control over any territories except those he directly inherited. Thus, the Habsburgs only controlled Austria and Bohemia, while Swabia, Bavaria, Brandenburg, the Rhineland and other important parts of the Empire were ruled by various independent dukes and margraves.

It is no coincidence that the Reformation was born in Germany. The various princes were so free from Imperial control that they could breakaway from the state Church at will. Charles V's efforts to stop the spread of Lutheranism only resulted in a bland acknowledgement of the status quo by his declaration of cuius regio, eius religio - a declaration that the princes alone would determine the religion of their lands.

After the Thirty Years' War, the Peace of Westphalia in 1648 codified the principle that the states of the Empire were almost completely independent from the Emperor. They could raise their own armies and form alliances between themselves and with other nations. The Imperial army was barred from passing through their territories without permission. Any new taxes could only be raised with the consent of the Diet. And so it went with most of the other structures of governance.

Your quote in the question is from Voltaire, who was writing in the middle part of the 18th century. At this time, comparing the Roman Empire of the Habsburgs the that of Augustus would have been a complete joke. The Habsburg Emperors had more territory outside the Empire than within it. They were in constant fear of being overrun by the states of the Empire, especially Prussia to the north. The religion of the Emperor was not shared by even half the German population. And the entire empire itself was less than fifty years away from its complete demise. Voltaire's quote was true then, but it probably would have been at any time within the previous five hundred years as well.

I am just going to add to legosim's comment. Like he said it was never run by the pope or any holy figure. There were numerous disagreements between the Emperor and the Pope. Emperor Henry VI was even excommunicated for a time over investiture. They also deposed Pope John XII in 955 and put a new pope in his place, starting a "Era of Three Popes".

The Holy Roman Empire was Roman in name only, like John and legosim said. They never even associated all that closely to the Roman Empire, they were calling themselves a continuation of the Carolingian Empire which also never called itself a continuation of Rome. The closest ties to it being called Roman, at least that I can find, was that the emperors were crowned by the Pope until the 16th century.

It was also more of a figurative empire than a literal one as it was a mish-mash of many different countries with an elected Emperor which held a largely symbolic title in the later stages. At their most centralized period the Emperor was simply a man that tied all the states into a large alliance. The largest of the states, Austria and Bohemia then later Brandenburg all usually could do almost anything they wanted save warring within the Empire. The Emperor could do little more than uniting the others against them, though that was a deterrent for some time, at least until Napoleon crushed them at Austerlitz.