State militias were called up in large numbers for the first time since the Civil War in 1898, when William McKinley realized that for the Spanish American War he needed significantly more infantry than he had in the regular Army. Initially it was thought the Army itself would be augmented by 60,000, but it was discovered the Army could not equip 60,000 more men, so a call was put out for militia volunteers. . Officers of the state militias, mostly hobbyists, also expressed a desire to fight as officers of their units, not get put into in the regular Army ( likely because they expected more advancement commanding their own units). But when Militia units actually presented themselves for the war, the result was close to a disaster. Many showed up for duty unfit: the states had not really trained or equipped them. They had old heavy wool or canvas uniforms unsuitable for the tropics, and were armed with the obsolete Trapdoor Springfield, and that took different ammunition than the Army's new Krag rifle. It was decided to place many of them in a few large camps for training- and then it was found that the ships, under the control of the Navy, were fewer than needed to transport the troops. There were delays. The huge camps, in places like southern Georgia, turned out to be excellent breeding grounds for disease, and quite a number of units suffered more deaths from things like typhoid than actual combat- if the unit ever saw combat.

There was as a result a major reform. Under the Dick Act of 1900 the old militias were scrapped and the National Guard created. Enlistees in the Guard were to be given up-to-date arms, equipment, and training, and their officers ceased to be hobbyists.

It is my understanding that this partially began during the debates on what would become the Barbary Wars. American merchant ships were being raided and their personnel enslaved by Barbary pirates off the coast of North Africa/Tripoli (reference to which was immortalised in the Marine hymn). New England merchants figured that paying insurance for piracy was a simply a matter of fact and refused to allow Jefferson and southern plantation owners to establish a navy, lest he become a tyrant, nor did they want to pay for the war through increased taxes. But Jefferson and his supporters thought it inconceivable that white Americans should be enslaved. The latter group eventually won the debate, a debate in which the beginnings of some abolitionist arguments began.

A few years later, the War of 1812 confirmed the need for a stronger and standing army. It wasn’t until the Civil War, however, when much larger conscription armies became a norm (almost akin to the kind of which Napoleon built up).

I’m sure others have more to add but this is hopefully a start.