I'd like to add to this question

I don't think France particularly held together better than the German states, rather it wasn't until around 1600 when France as we know it, consolidated. Prior to this, and following Charlemagne, France had it's many pieces, independent Duchies and Counties that generally were under varrying vassalage to the Crown. Normandy, Aquitaine, Navarra, and Burgundy being the biggest examples of this, who maintained decent independence throughout their lifespans.

But the German states, east of the Rhine, were definitely more fractured, and quasi-independent in comparison to the region of Gaul. so my addition, is wondering if these differences are primarily because France was ousted from the Holy Roman Empire. I think the difference likely lies in the events that took place. Royal marriages and such, probably unified German states to less effect than those in Gaul.

When it came to establishing central control, the Holy Roman Emperors had several disadvantages compared to the kings of France.

First, the HRE was much larger, covering much of Italy as well as Germany. Any would-be Italian ruler was likely to get into conflict with Italy's other preeminent force, the Pope, which meant that rebels against the Emperor could get Church support. The HRE's size also meant that a new Emperor might find himself playing whack-a-mole trying to establish his authority. When the Emperor was busy enforcing claims in Italy, he was a long way away from the princes in northern Germany.

Second, the early Emperors relied heavily on Church officials to control the Empire. When reforming Popes moved to assert their authority over bishops and archbishops -- in the investiture crisis in particular -- they cut out a major prop of the Emperors' control over their administrators and several key states, and triggered rebellions that ended with parts of the Empire, notably Saxony, all but autonomous.

Third, and perhaps facilitated by the first two, the crown remained elected, not simply inherited. While sons often followed fathers, at other times the Electors, princes of the realm, could use the election to choose a weak king, or demand concessions from a would-be king, or to legitimize a rival king if they wanted to rebel. Even without election games, a change in ruling families meant new people who expected rewards and old officials whose loyalty might be suddenly less secure. None of that helped centralization. It took until the 1400s for a single family, the Habsburgs, to get a lasting lock on the kingship. By then, the French had spent 500 years under kings whose father or grandfather had also been king.

Those factors perhaps explain why the Empire's centralization lagged behind France before 1500. After 1500, three new factors come in to aggravate the situation: the Habsburgs' external commitments, the Protestant Reformation, and the rise of Prussia.

Of course, the Habsburg Emperors would have liked to unify the HRE. But the Habsburgs acquired so many territorial claims -- and antagonists -- outside the HRE that the old problem of divided attention reared its head in new dimensions. Oh, the Habsburgs weren't laggards about assembling land and power, but lots of that was land and power outside Germany, or even outside the HRE. At the same time, the French themselves, seeing a need to fight Habsburg power, gain a motive to subsidize division on German soil, most famously backing Protestants during part of the Thirty Years War.

Which brings us to the next early-modern factor undermining HRE unifcation: the Protestant Reformation. For centuries in Europe, kings perceived religious unity as vital. Once substantial portions of the HRE were Protestant, the Emperor had every motive to enforce Catholicism where he could get away with it, and the Protestant states had every reason not to let the Catholic Emperor acquire that kind of power over them.

Finally we have the Prussians, who emerge in northern Germany as a strong state counterbalancing Habsburg Austria in the south. By the time growing religious toleration makes a unified Germany more conceivable, the inability of either the Prussians or the Austrians to knuckle under to each other makes it less feasible. In the end the matter isn't resolved until the 1800s, when Bismarck contrives a war that gets the Austrians out and leaves the Prussians dominant.

So to sum up, the HRE was dealt a perversely unlucky hand by being "too big" and by relying on Church officials right before a wave of Church reform came in. That kept centralization back long enough for other factors to arise and keep it back: elective kingship, Habsburg foreign commitments, the Protestant Reformation and the rise of Prussia.

Having said all that, it's worth remembering that the Holy Roman Empire worked pretty well for its citizens for centuries. Why should a city merchant regret his Emperor being less than a dictator, so long as the courts defend his city's rights and trade is good? It's not clear that the Germans were worse off for not having the unity of the French, at least until the Thirty Years War.

Sources: The New Cambridge Medieval History, v4&5, chapters of Hanna Volrath, Benjamin Arnold and Michael Toch; The Sicilian Vespers, Steven Runciman; Iron Kingdom, Christopher Clark; the Thirty Years War: Europe's Tragedy, Peter Wilson. I'm an amateur, so please correct freely and with confidence.

It comes down to how each formed; Germany started off as a kingdom more centralized by its major monarch, who would go on the form the HRE: Otto I. He consolidated the realm, but when the political situation in Italy broke down he was appealed to by the Pope and widow of the former king to move in to help, which he did because Italy was wealthy and there was political/religious prestige to resolving the issues there (he was also trying to use the clergy in Germany to solidify his centralizing reforms too and getting papal blessing was hugely helpful). So he married Adelaide of Italy and had several children, one of whom would inherent both the Italian and German thrones, while consolidating North Italy politically; the Pope of course did not like having a German king of that power staying around in Italy and ended up getting deposed by Otto and his replacement crowned Otto as emperor, inheritor of Charlemagne's crown with Papal blessing and defender of the faith.

The problem of course was that Italy was culturally and linguistically different from Germany while also being separated by the Alps and the Pope now had a vested interest in the HRE. The Italian nobility constantly rebelled forcing the HRE Emperor to have to campaign in Italy leaving Germany politically open to rebellion because there was not an Imperial authority strong enough to keep the German nobles in line. Plus after Otto deposed the Pope, the Papal authorities were interested in reasserting their power as the dominant institution, so were constantly interfering in HRE politics and supporting revolts against the Emperor to keep him off their backs and politically subservient.

So unlike France Germany was linked to other states that were not happy to be part of the HRE with Germany and religious interference into the workings of state. Under a strong monarch that was somewhat manageable, but not all monarchs were strong, some dying before their heirs were older than 5, meaning the forces pulling apart the state were not counterbalanced by a centralizing force. The elector system evolved out of this and required the Emperor to buy their approval for heirs, while the various hereditary lines kept dying out early on, much sooner than the French monarchy in the Middle Ages, so Germany evolved a decentralized system, while France evolved a strong hereditary monarchy. Germany was interfered with so much early on to keep it weak due to all of the political interests wanting to keep Imperial authority from centralizing.

If we play a counterfactual and the HRE never formed under Otto, say because Italy was more stable, then Germany would have actually formed a strong centralized hereditary monarchy under Otto as just the Kingdom of Germany that would then interfere in the politics of her neighbors rather than the other way around and the Pope would have little reason to get involved in German politics like he historically did and things would evolve in a 'French' direction sooner than France would have. But it was the desire for empire and prestige that set Germany down the path of political fracture and decentralization.