The short answer is that the Vichy government was the legitimate government of France, while the Free French were a bunch of traitors. The longer answer requires a bit of knowledge about the terms of the French armistice.

First, some background about the situation. With the capture of Paris, the government of the third Republic fled the city for Southern France, eventually settling in the town of Vichy, France. At this point, the French could still have arguably continued the war given the size of their colonial empire, the strength of their remaining units inside France (e.g., the many divisions of the Maginot had yet to see combat), the strength of their army outside of European France (e.g., in Algeria), the fact that the Germans had exhausted their supplies and outrun their supply lines (e.g., they were over-extended), and the fact that the British were landing a second expeditionary force in France after the debacle of Dunkirk. However, the speed with which the French army had collapsed on the front, combined with how complete the collapse had been, completely destroyed the will of many of the members of the French government to continue to fight. Although many members of the government wished to continue the fight, the members in favor of the armistice won the debate.

​

So the key here is that the lawful government of France, having relocated to Vichy, sought an armistice to end the war. When the French Third Republic agreed to the armistice terms, the French Third Republic did not cease to exist. As a result, they had the legal authority to instruct all French units to stop fighting. The French Navy and other Vichy Forces remained loyal to the Vichy government because it was still the legitimate, constitutional government of the Third Republic. To disobey a lawful order received through the legal, established chain of command would have been insubordination at best, and treason at worst. So while the Vichy military had no love for the Germans after the armistice, they weren't about to commit treason to France either. As the historian Max Hastings wrote "For us Frenchmen, the fact is that a government still exists in France, a government supported by a Parliament established in non-occupied territory and which in consequence cannot be considered irregular or deposed. The establishment elsewhere of another government, and all support for this other government would clearly be rebellion." This is why most of the French military sided with Vichy France during the first part of the war.

​

There's also a second part of your question, why not just join the Free French? Other than the treason part, the Free French were pretty much a rag-tag band of misfits until the end of 1942 / beginning of 1943. When De Gaulle fled to England and announced that he would keep fighting, he was essentially a general without an army. After the fall of France in 1940 until 1942, De Gaulle convinced some colonial units in the most distant parts of the French Colonial Empire to join (primarily sub-Saharan French Equatorial Africa in the region of Chad and the Congo), allowing him to establish the Free French capital in Brazzaville, Congo (across the river from Kinshasa in the Belgian Congo). However, they were few in number with a mostly ineffectual military record. In other words, this was not exactly an awe-inspiring alternative.