Hypotheticals are always a bit tricky for historians. There are many different answers for "why x did why instead of z" and these answers can generate more heat than light. But while there were fears within the Habsburg war party that Napoleon would dismember the empire, there were solid reasons weighing against Napoleon following this course of action.

For one thing, it bears mentioning that Austria did not come off lightly in the 1809 war. It lost significant territory within Galicia and Dalmatia. The latter territory became the French Illyrian Provinces and effectively landlocked Austria. Napoleon kept the provinces under direct French administration but also dangled their return as a guarantee of Austria's continued amity. The losses of territory, around three million subjects, as well as a sizable indemnity meant that Austria was, in the words of Michael Broers, "neutered" for the rest of the Napoleonic Wars. Austria would not act unilaterally against Napoleon between 1813-15, it was always within a coalition system whose allies could put substantial boots on the ground.

But Austria did act against Napoleon despite the inequities of the peace treaty, so this does beg the question as to why Napoleon did not press further in his peace claims. Total dismemberment though was not a viable option. Napoleon did pursue this tactic against lesser and middling states such as the Dutch and Italian states. But they were considerably smaller than the Habsburg Empire. Spain was the largest state Napoleon tried to impose a complete takeover by forcing the Bourbons to resign and placing his brother Joseph on the throne. The Spanish situation in 1809 had yet to develop into the "ulcer" of later years, but even at this early date it was becoming clear Spain was not proceeding as Napoleon had anticipated. Replicating Napoleon's Spanish gambit in Austria would have been even more difficult. For one thing, Napoleon was running out of reliable family members that he could install in Vienna. Napoleon's troubleshooter, Joseph, was busy in Spain. Both Louis and Jerome had disappointed him in 1809. His sisters lacked the political capital for such a post. Placing a marshal or ally as executive head was fine for a smaller territory like Illyria or the Duchy of Warsaw, but not a major power.

The lack of potential replacements for the Habsburgs might have ruled out a dismemberment, but there were also concrete reasons for pursuing the peace he did. One of these reasons was the continued existence of the Austrian army under the command of Archduke Charles. Wagram had been a victory, but not a decisive one in the style of Austerlitz or Auerstadt. Napoleon would say of Wagram as it was closing "War was never like this. Neither prisoners nor guns. This day will have no result." Charles may have felt beaten, but he retired in order and there was no close pursuit by the French and their allies. The armistice at Znaim had been given without Emperor Francis's permission, but it preserved a sizable Austrian force that would have counteracted a Carthaginian peace by Napoleon.

More worryingly than Charles's battered army was the complete failure of the Russian alliance in 1809. The Tilsit system had envisioned Franco-Russian cooperation against Austria, but Alexander I was hardly an enthusiastic ally. Russia did mobilize against Austria, but its mobilization was tardy and Russian Army generals showed an open desire to fight their "allies" among the Duchy of Warsaw than the Habsburgs. This showcased a cardinal failure of the Tilsit system and underscored to Napoleon his lack of a solid great power ally within Europe. If Russia was an inconstant ally, the only real option left for Napoleon was a rapprochement with Austria.

With these factors in mind, Napoleon probably got the maximum peace he could get from Austria in 1809. Although hindsight shows how his attempt to bind Austria to him was a failure, Napoleon's tack towards Austria almost bore fruit. Unlike Prussia, the Habsburgs and their ministers kept an extremely tight rein on war parties aching for revenge. Neither Austria nor its army unilaterally turnded their cloak in the immediate aftermath of the 1812 debacle. Prussia's army did do that, fatally undermining the wreck of the Grande Armee in Poland. Instead, Austrian diplomats kept to a guarded neutrality and offered mediation instead of war. Metternich was willing to deal with Napoleon as a legitimate head of France all the way through 1814 and Austria remained the most pro-Bonaparte of all France's enemies. Marie-Louise's marriage certainly helped here as Metternich increasingly championed a regency and Napoleon's abdication. But Napoleon always overestimated his capital with Austria and underestimated the costs which Austria demanded as the price of a pro-French foreign policy.

Sources

Broers, Michael. Napoleon. Volume 2, he spirit of the age, 1805-1810. London: Faber & Faber, 2018.

Mikaberisze, Alexander. The Napoleonic Wars: A Global History. New York: Oxford University Press,2020.

Rothenberg, Gunther Erich. Napoleon's Great Adversary: Archduke Charles and the Austrian Army, 1792-1814. New York: Sarpedon, 1995.