Naturally exactly what happened depends on the country you are talking about.

However, on that note I believe that one of the most interesting examples to discuss is that of France. As you probably know, France and Germany had been at war with each other since 3 September 1939 and, though the fighting was somewhat limited compared to other theaters, there were significant casualties on both sides. In fact, though France had suffered a crushing strategic defeat during the Battle of France, its military was not completely destroyed. Even through 22 June 1940 when the armistice between Germany and France was signed, there were factions of the French government and military which favored continuing the war. For example, as late as the 3rd of June, Prime Minister Paul Reynaud asked Admiral Darlan to study the possibility of evacuating to Algeria in order to continue the fight. To this end a number of members of France’s government left for Algiers only to be arrested on arrival after the political climate had changed during their journey.

This change in climate was heralded by the assumption of full powers by Marshal Petain. In the belief that France was fully defeated and needed to save itself in preparation for a national resurgence and revolution, Marshal Petain brokered a painful armistice between Germany and France. He cast himself as sacrificing himself for the future of France and said that “the French renaissance will be the fruit of this suffering . . . I will refuse to leave metropolitan soul and will remain among the French people to share its suffering and misery. The armistice in my eyes the necessary condition of the durability of eternal France.”

As such, much of Petain’s political effort went towards preserving the basics of French sovereignty. Though the military of France was greatly reduced in size and subordinated to Germany, France was still allowed to retain about 150,000 men under arms by the terms of the armistice. These men were tasked with keeping order in the non-occupied zone of southern France and guarding French soil against allied incursion. They were later supplemented by an unarmed militia headed by Pierre Laval which operated as a sort of secret police for the Vichy state and German interests. Nonetheless, the Vichy government took pains to portray itself as at most a co-belligerent with Germany and not an ally.

One of the most powerful tools that France retained in order to express this was the French fleet. It was outside the direct control of German authorities, and as such was a powerful tool of negotiation. Included in the armistice was a provision which stated that Germany "solemnly and firmly declared that it had no intention of making demands regarding the French fleet during the peace negotiations." The presumption among the French was that their fleet would remain theirs so long as it remained in harbor. Along with its colonial possessions and other political concessions Vichy tried to use its remaining resources to secure the release of the some two million French prisoners of war who were interned in Germany. However, Vichy’s maneuvering came too little as only a fraction of said prisoners were released during the war.

Ultimately, what little sovereignty the Vichy French government and military had was eroded over time as the fleet was attacked by the British at Mers-El-Kebir, colonial military bases and airfields were used to support German efforts in Iraq, Syria, Algeria and Tunisia, and other overseas territories like Senegal were occupied by the Free French Forces under De Gaulle. Additionally, some French men were recruited to go fight Bolshevism in the Soviet Union under the banner of France and Germany with the Légion des Volontaires Français contre le Bolchévisme. However, this militia did not have any formal ties to Vichy. Eventually when Germany moved to occupy all of France following the fall of Algeria the French fleet was forced to scuttle itself at Toulon in order to prevent its capture.

In the case of continental France, the military was effectively disbanded. What remained was essentially a police force for battling resistants and maintaining security. It was hugely inferior to the German occupying forces and was restricted in its equipment. However, continental France does not tell the whole story. As alluded to previously there was a noticeable contingent of the French military which favored continuing the fight. This group later coalesced under General Charles De Gaulle who formed the Free French forces. He was able to establish himself in colonial French Africa and was also able to exert influence in continental France through his efforts to organize the varied French resistance groups under the umbrella of the Free French Forces of the Interior. In this sense, some aspects of the French military never stopped fighting, and would in fact return to France after D-Day as a component of the Allied armies.

This is mostly taken from Julian Jackson’s France: The Dark Years 1940-1944. Also recommended would be The Strange Defeat by Marc Bloch.