Okay, so I hate to start off on a negative, but I must address a problem with your question. Having a large army does not make a country "militarized", especially when that large army is a conscript army that is only raised in times of war. You could make a case that France was a militarized state if they kept the army that big in peace time, but remember that the army only got that big because France was literally fighting for its life against Germany. A militarized society, to me at least, is a state that's politics and society is dominated by the military; a state where the military is put on a different platform than the rest of the government.

Now to get to the actual question as to why we consider Germany to be such a militarized state, well that largely comes from the fact that the most powerful and most dominant country in the German union was Prussia. Prussia had acquired a reputation for being a state dominated by its military; so much so that it was once quipped that:

Other states possess an army; Prussia is an army which possesses a state

But it should be noted, that while there is no debate that Prussia/Imperial Germany was a militarized state (the army tied up a huge amount of the state's resources), there is debate as to how much of a militarized society Prussia/Imperial Germany actually was. I do believe that they can certainly be classified as a militarized society, but their society wasn't completely militarized; and I feel that by putting this militarism label on Prussia/Germany people overlook the other non military related accomplishments of the state as well as the fact that both states had a healthy non militaristic side to their culture. That being said I will explain the points in favour of Prussia/Germany being both a militarized culture and a militarized state.

Prussia certainly deserves its reputation, under Frederick the Great, the Prussian Army numbered around 200,000 men, which made it the fourth largest army in Europe. At the time Prussia wasn't even in the top ten in terms of population or land area. Under the reign of Frederick the Great's father, Frederick William I, the Prussian Army consumed around 70% of the state budget. Now these events were long before WW1, but it helps one understand why the world thought of Prussia and by extension Germany as a militaristic nation. Prussia was seen as nation where everyone lived for the military and a nation where military culture was so prevalent that everyone blindly obeyed the military. A good example of this, is the case of Wilhelm Voigt. Its a long story, so to give you the short of it, Wilhelm Voigt was an old robber and swindler. He acquired a German officer's uniform and used it to fool some soldiers into helping him rob a bank. (they stood guard while he used a forged receipt to acquire the money) The world press, especially the French press had a field day with this incident, and while Voigt was eventually caught, it was a deep humiliation for the German army and the German state. The world saw this as proof that the Germans would blindly obey anyone in a military uniform, one newspaper even said that this type of thing "could only happen in Germany".

Now one has to wonder, does Germany actually deserve its reputation as a highly militarized state? The Army played an absolutely huge role in German society, specifically those areas that had once been apart of Prussia. There were annual festivals that commemorated the defeat of France at Sedan in the Franco-Prussian war, and other important victories were celebrated. Marching bands, military parades, and shows where the German military would preform maneuvers for the public were all common. In any sort of state parade or festival, the military would play a key role, meaning that the images of war and the military would always be front and centre in the public mind. In garrison towns (towns with a military detachment stationed in the town), the military would play an even bigger role in public life, as those towns often revolved around the military barracks and they depended on the military to sustain the town.

The cult around the army even affected the German education system. The textbooks heralded the army as a noble institution, and success in the army was seen as one of the greatest achievements one could hope for. Major focus was given to great battles in German history, and the role the military played in the wars of unification. One German bureaucratic charge of reviewing the textbooks, had to ask: "Is this a history book or a manual of war?". Certian schools taught children the letters of the alphabet by associating the letters with military objects:

When pupils came to the letters ‘P’ and ‘K’, the list of associated objects that they encountered included ‘powder’ (Pulver), ‘bullet’ (Kugel), ‘cannon’ (Kanone), ‘cannonier’ (Kanonier), ‘king’ (K¨onig), ‘arrow’ (Pfeil), ‘pistol’ (Pistole), and ‘cuirassier’ (K¨urassier)

Patriotic celebrations were held in schools, and to use the words of historian James Retallack:

The goal of patriotic celebrations in the schools, as one Prussian official explained in 1888, was to cultivate in children a ‘powerful stimulus to emulate their fathers’, the veterans of the war of 1870–71, who had ‘risked everything to ensure the highest earthly blessings for their descendents’.11 The songs best designed to provoke sentiments like these, counselled a school director in 1912 on the basis of his own experience, were set to march-time and had ‘fresh patriotic and enthusiastic military content’.

For many families, displaying pictures of their children in military uniforms was their biggest source of pride. These kinds of measures may seem small, but they are indicators of just how important the military was.

Furthermore, one has to look at how much influence the German Army had over the government. The army depended on the Reichstag for funding, but in all other cases it was controlled by the German general staff, who in turn reported directly to the Kaiser. The German army's popularity surged because of the wars of unification. The wars against Austria and France in which the Prussian led armies of Germany absolutely decimated their opponents, allowed the military to put itself on this pedestal outside of Government control. To give you an idea about the power of the military, one only needs to look at the Hereto rebellion. The local German governor of German South-West Africa was having issues suppressing the rebellion, and he requested reinforcements from Berlin. The Kaiser responded by sending Lieutenant-General Lothar von Trotha. Now in theory both men should have worked together, and Trotha should have been subordinate to the German Governor. But since Trotha was part of the military, he answered directly to the German General Staff, and thus the Kaiser. The German Governor had no authority to stop Trotha who resulted to absolutely brutal methods to suppress the rebellion. It was a clear indicator of the power the military had, and the inability of the German Government to reign in the military.

The German Army's control over German politics would later be emphasized during WW1, when two generals, Paul Von Hindenburg and Erich Ludendorff became virtual dictators of Germany by virtue of being the leader of the German general staff. Paul Von Hindenburg was in command of the German 8th army which won a massive victory against the Russians at the battle of Tannenberg. These victories made Hindenburg a national hero, with an enormous cult of personality around him. Now Hindenburg and his chief of staff, Erich Ludendorff (who held the real power and used Hindenburg as a puppet) argued that German should focus more on the Eastern Front. The chief of the German General Staff Erich Von Falkenhayn, argued that the war should be fought primarily in the west. Eventually, Falkenhayn got discredited due to his inability to make any progress on the Western Front and Hindenburg was made Chief of the General Staff. Ludendorff held the real power, and using his influence and power was usually able to control the Kaiser and Hindenburg. This in essence made the Army the unseen ruler of Germany. The fact that a military general had so easily been able to take control of the German government, showed that the Army was without a doubt the most powerful organ in the German Empire.

So, the German Empire can definitely be considered a "militarized state" because of the preeminence of the military in the social and political life of Germany.

So some good books to read on this subject are "Imperial Germany: 1871-1918" by James Recktall, "Iron Kingdom, The Rise and Fall of Prussia: 1660-1947" by Christopher Clark, and "A Nation in Barracks: Conscription, Military Service and Civil Society in Modern Germany" by Ute Frevert.

And as a follow-up question, why did Austria-Hungary have such a tiny army, half of France, with similar population?

Because Austria-Hungary was stuck in the past in a lot of areas and stuck in multinational issues in others.

In the Ausgleich of 1867, which parted the Austrian Empire into Austria and Hungary, the army suffered dearly. Because the Kaiser had to grant the Hungarians their own army, the Honvéd (or k.u. Landwehr) he effectively had to grant one to Austria as well (the k.k. Landwehr) - even if they were formally just local militias since the proper army (Gemeinsame/k.u.k. Armee) was one of the three joint ministries.

So officially Austria-Hungary had its army, supported by the two individual standing Landwehr militias and general reserve militias (the Landsturm). Though in reality these different armies had vastly diverging priorities. Funding the Gemeinsame Armee was unpopular everywhere since the country had essentially never been in a proper war, the last one being the Austro-Prussian War in 1866 that lead to the Ausgleich in the first place. Hungary, who had in the Ausgleich demanded ther own army, had quickly turned their infantry-only militia into a properly combined forces army, with Austria later following suit. At times, the Landwehr was even prioritised over the Gemeinsame Armee in supplies and manpower - for example the k.k. Gebirgstruppen (mountain division) of the Austrian Landwehr were by far the best-trained and equipped of any Austro-Hungarian forces.

Tactics and strategies too were antique, with Chief of the General Staff Conrad von Hötzendorf sticking to an outdated policy of "no grounds given", leading to needless loss of lives over strategically useless land early in the war. Amongst other things. It's worth noting though that Franz Ferdinand was working on improving the situation and in 1913 was actually made General Inspector of the Army and given full support to modernise the forces. Unfortunately, he couldn't do a lot in that one year.

The military industry as well was weak and not suited for mass production, so it could never keep up with the demand. Most of the regular armies had at the start of the war received contemporary rifles and artillery (although many still lacked proper uniforms, most of the cavalry moved out in shiny parade gear), but there were no leftovers for the much needed Landsturm since the military administration had a long history of "let's not throw that away, it was expensive and we might use it one day".
When the Italian front opened in 1915 you could regularly encounter Landesschützen (riflemen recruited from and for the defense of Tyrol) dressed in civilian Tracht with just a yellow brassard marking them as soldiers, carrying either their own rifles or the supplied Werndl rifle, a single-shot rifle as old as the country itself. Even worse, due to the government right at the start of the war having taken three regiments of Landesschützen to support the Russian front, the lack of experienced locals led to desperate quick-training of disabled or otherwise unfit people. So not only would you regularly see what was essentially private people "helping out", the age range anywhere between 14 and 80, some of those were made officers with little to no experience or service time, leading to constant hate and disregard from officers of the regular army.

TL;DR: Peace sometimes is worse for a proper army than war.