This question was the centerpiece of a huge debate at the time of German unification - the debate between the "kleindeutsche Lösung" (~small Germany solution) and the "großdeutsche Lösung" (~large Germany solution). "Small Germany" being Germany unified with Austria left out, and "large Germany" being Germany with Austria included.

The root of the conflict stems from the upheaval brought by the Napoleonic wars that dissolved the Holy Roman Empire (of the German Nation), the former overarching political structure of the German region. The Austrian Habsburg dynasty had controlled the HRE for a long time and wasn't willing to let it go completely, so they pretty much retained the title of Emperor for the state of Austria-Hungary itself.

Now, at the Vienna conference after the final defeat of Napoleon, the victors decided on a new political structure for Germany: The "Deutsche Bund" (German Confederation). In the German Confederation, Prussia and Austria were the leading powers with Austria being the nominal head of the confederation.

This is where a lot of the controversy stems from: Neither of the powers was being ready to submit to the other, a conflict that would cumulate in the "German War" (as it is called in Germany, while it's called Austro-Prussian or Seven Weeks War in the rest of the world) of 1866. Now, Austria was not without allies in Germany proper: Most of the "middle states" (medium sized states sandwiched between Prussia and Austria, such as Saxony or Bavaria) were fearful of Prussian expansion and sided with Austria in hopes of support. The "big Germany" solution thus was the solution in which Austria would retain the lead in Germany and grant extensive federal rights.

The Prussians on the other hands favored a more centralized approach and weren't all that much for extensive federalism, founding the "Zollverein" (German Customs Union) in the early 1830s. This Union was primarily aimed at standardizing weights, scales and such and lifting customs at the borders of member states, making transportation of wares and trade in general much cheaper and easier. But it also cemented the Prussian hegemony over the member states, since the Prussians gained extensive rights and the member states had to give up important state rights, becoming client states in all but name.

Also not helpful was the problem of Austria not being only Austria, but Austria-Hungary. Austria-Hungary was by no means Austria with Hungary as a nice little vassal, but rather a personal union. The Hungarians had gained extensive rights in this union over the time and were not without say in it. Also, Hungarians were not German and were not included in the German unification movement or forming German nationalism. Proponents of German unification with Austria included always retained that Austria would have to dissolve the personal union and leave Hungary excluded from Germany. But Austria was not really ready to just give up the larger part of their territory, just to become leader of a Germany that they considered to already control via the German Confederation anyway.

So, Austria was not really invested into the whole German unification thing anyway. And while Prussia wasn't really either, the King of Prussia vehemently refusing to become German emperor (even though being offered the crown by representatives of the 1848 revolution), Bismarck saw in the growing unification movement a perfect vehicle of ensuring Prussian hegemony over all of Germany and of defeating the ancient rival Austria once and for all.

Thus Bismarck stood behind the "small German solution" and pushed for it, leading to the German war (/Austro-Prussian war) and the exclusion of Austria from the unified Germany.

The other post has the big of it, but the Wikipedia has a weirdly extensive article on the small of it. After the nominal end of the HRE, you have two significant "German" political powers, the Hapsburg monarchy and Brandenburg-Prussia. One cannot forget that Prussia was very much involved the War of Austrian Succession, and were definitely not interested in restoring a Hapsburg emperor.

Aside from the dynastic-political consideration, you have more-Catholic Austria and the more-Protestant north German states. There was also some question of the status of Hapsburg-Austrian territories which were not German, given the ethnonational aspects of this whole process.

Austria's poor showing in the Austro-Prussian War closed the issue. If the lesser German polities had to make a choice between the two powers, Prussia was the more advantageous ally. And that was it.