A substantial element of Austria's separation is the Treaty of Westphalia, which concluded the Thirty Years War. The Protestant powers of Europe wished to abate the Hapsburg dominion by guaranteeing the freedom of the various elector princes to decide their religion.
There is also the role of the House of Wittelsbach, rulers of Bavaria, who were long-standing rivals to the Hapsburgs. As the HRE coalesced into the Hapsburg dominated Austro-Hungarian empire, Bavaria stood against them.
So while linguistically homogenous (somewhat), there has been substantial religious, cultural, and economic divergence between Germany and Austria.
In many ways, your question is akin to asking why Canada isn't a part of the USA. Except that there have been far more wars fought over Germany not being a part of Austria than there have about Canada not being a part of the USA.