Well, most of the colonies initially decide to go with Vichy, for a variety of reasons. For one thing, as 13th Airborne notes, Free France did really just look like a lot of people in London, and there were a lot of people speculating in June of 1940 that the war was effectively over. Beyond momentary convenience however, consider too that the people who come to power in Vichy France (such power as they were allowed to wield) were pro-empire. Pierre Laval, who was Prime Minister from 1942 onward, had also been the minister of the colonies, and there were a lot of French colonial administrators not unsympathetic to the Vichy vision of what France and its overseas territories should be.
If you read Vichy in the Tropics by Eric Jennings, he makes it clear that many administrators who refused to defect to the Free French saw the new administration as an opportunity to try and realign native nationalists into a greater French Empire. Their attempts pretty well backfire in places like Madagascar and Vietnam, where in the latter so-called nationalist clubs ended up being recruiting places for the Viet Minh, and at least it's not coincidental that both colonies see uprisings in the postwar. But that doesn't mean that they immediately meet violent resistance either. Aime Cesaire becomes anti-Vichy in 1941, but it's not an automatic response; he has to be provoked a fair bit by the patrochialism of the Vichy governor, and as Jennings notes, a lot of people on the island looked back fondly on that time as a period of self-sufficiency, despite the fact that they eventually found themselves living under a blockade.
Security also enters into the equation when colonies made the choice to declare for a side. The French possessions in India almost immediately declare for De Gaulle, but they're completely surrounded by British India. Not only is there no risk in siding with Britain, there's arguably much greater risk in not doing so. Some of the Pacific Island possessions go along for similar reasons, mostly because of their proximity to Australia. Then look on the other hand at Indochina, which is cut off from the metropole and situated dangerously close to an openly acquisitive Imperial Japan; they in effect become puppets of the Japanese government until March of 1945, when the Japanese assume formal control and dispense with the French. For the French administration, there wasn't much of a question about which side to take if they wanted to stay in any kind of power. Embers of War takes a quick look at this (and the First Vietnam War). It's not the be-all, end-all consideration: France's larger colonies in the Americas remain under Vichy control into 1943 or 1944 despite being pretty far removed from the Axis powers, but it definitely plays a role.
According to Eisenhower's memoirs of the war years (Crusade in Europe), there was no question. Vichy France was the successor state of France. Free France were a group of people (mostly in London) with will and ambition, but nothing else, not even unequivocal support of the allies.
The book "Eisenhower in war and Peace" (by Jean Edward Smith) explains the politics of the US relations with the French, particularly with regards to planing the invasion of Africa. All the French colonies in Africa were controlled by people who chose to support Vichy as the only legitimate remainder of France - some of them viewed collaborating with the British as no better than collaborating with the Germans. Particularly galling was the sinking of the French fleet at Oran (done by the British so the fleet would not fall into German hands - but resulting in large French casualities).
Some of the US diplomats favored coming to an agreement with Vichy representatives, because they were seen more (Catholic) pious, while the Free France were more secular.
According to both "Crusade in Europe" and "Eisenhover in War and Peace", Free France only gained strength when the allies took over (mostly with diplomacy, but also after some hard fighting) French colonies in Africa.
Eisenhower was loathe to leave Vichy sympathizers in control behind his front-line, so coordinating with Free France to replace the unreliable people with those who were leaning to the Allies was essential.
Though the books describe at least one case where Free France assassinated a French leader Eisenhower found useful. However, Eisenhower and De Gaulle developed a good working relationship which became vital in the planning and executing of operation Overloard and persisted after Eisenhower left the military.