Essentially, it boils down to Austria-Hungary being in a situation where they had too many internal issues, while being a conservative nation that had historically, prior to the 19th century had no major territorial ambitions overseas. The most that could be considered a colony was a short-lived attempt in the Indian Ocean, a brief control of an East India Company based out of the Austrian Netherlands, both brief and during the time of Maria Theresa. Basically, the reasons the Indian Ocean colony (on the Nicobar Islands) failed was sheer distance, an insufficient initial quantity of colonists (only sixty, and quite ill-equipped for the task), and a lack of trade goods. Further the Austrian colonization was essentially an attempt by opportunistic merchants to pick up the pieces of a failed Danish colonial effort, and not an independent effort of their own. There was a second attempt later on, but it was a purely private commercial venture based out of southern China by an Austrian merchant who wanted to establish a colony in Borneo. It failed due to competition from a lack of funding after a number of years, as well as total disinterest from Vienna. Now, this basically turned the Austrians off to any sort of wild traditional colonial attempts overseas.

However, it did not mean that Austria had no territorial ambitions during this period. The difference is that with a weakened Ottoman state along its southern borders, Austria's ambitions could also be sated much closer to home. Its political outlook, especially prior to 1870, with the Ausgleich of 1868, and the rise of Prussia into the German Empire after 1871, was all about maintaining the status-quo, and not upsetting it on the Continent. After these twin events, the Compromise which shook the foundations in many ways of the Austrian state, and the rise of Germany as a unified nation upsetting the balance of power that Austria was so fond of maintaining, it became increasingly interested in a new round of territorial expansion. It was really over after these dual events that Austria saw her prestige waning and took any sort of interest in going abroad. However, the Austrian state was also a highly centralized and incredibly complex government that simply was impractical for the sort of distances that colonial empires would demand. Thus in order to be able to effectively govern the territories they would take to maintain prestige-relevance, their territorial acquisitions had to be essentially contiguous with their existing centralized nation.

I touch somewhat on the Nicobar attempt in this first answer I wrote some time ago, though it largely deals with the issue of language in a hypothetical Austro-Hungarian colony. I also give a wider and more detailed examination of Austro-Hungarian (dis)interest in the Scramble for Africa in this second answer.

EDITS: Added the second paragraph when I realized it didn't go through earlier.

In a way, they planned to handle the colonial race and the age of Imperialism the same way Russia did; expand at the cost of their neighbors instead of expanding on other continents. This approach favored Austria-Hungary and Russia since it didn't require a large navy and trading fleet, merely roads, infrastructure and an army.

However, whereas the Russians succeeded in expanding and expanding right up to and even beyond the First World War, the Habsburgs essentially stagnated after the Congress of Vienna in 1815. Despite being ultimately on the winning side of the Napoleonic Wars and gaining Lombardy, they had lost lands in Poland and the Netherlands, and their influence was waning due to Prussia's ascendancy. They would lose Lombardy and Venetia in wars with the Italians, but would later gain Bosnia. Serious ambitions in the Balkans were thwarted by Russia, who also sought to expand there, and the British and French, who wanted to keep the Ottoman Empire strong there as a counter-balance to Russia. Prussia and later Germany tried to keep the status quo in the Balkans, which meant they also hindered Austrian efforts to expand there. When Austria-Hungary did annex Bosnia, it triggered the Bosnian Crisis, an event many consider to be one of the crucial building blocks toward the eventual Great War a few years later.

The Habsubrg Dominions, later the Austrian Empire and later the Austro-Hungarian Dual Monarchy, was a multinational state, and some nationalities had more rights than others, most notably the German and Magyar populations, although Poles also had notable autonomy. This was countered by other nationalities, notably the Serb, Romanian and Ruthenian populations. This is comparable to Colonial Empires having a caste system and favoring certain local groups over others, and also to the Russian Empire, which was also a multinational state covering a vast area.

To answer the second part of your question, Germany suddenly gained colonial interests primarily due to big business wanting to use colonial exploitation, but also colonialist lobbies who felt that Germany, as a great Empire, should have a large colonial presence and a large navy to back it up; Bismarck was opposed to such endeavors, which is major factor to Germany not getting involved until fairly late in the race, but another factor was that Germany as a nation didn't exist until 1871. Wilhelm II and his Weltpolitik new direction was led to Germany suddenly having colonial ambitions.

Also, fun fact, Austria-Hungary, after taking part in the Eight-Nation Alliance in the Boxer Rebellion, was granted a concession port in Tianjin, China, and ruled it for 16 years.