What is the difference between an annex piece of land and a colony? When France took control of Corsica it was apart of France, when it conquered Madagascar it was a colony. When England had control of most of France it was annex territories, when it had conquered Egypt it was a colony.
I thought it was distance but France had Algeria and that was close to the French mainland. Then I thought it involves them the two parties being from different continents, but Japan had colonies in Asia and the UK had Malta which was called a colony. Then I thought it was when they try to supplaint the native population with their own. I thought that because of how the native people of the Americas and Australia/New Zealand where supplanted by Europeans. But in every other example other then the Americas and Australia didn't have the native supplanted by the colonizers.
One thing you might want to consider, is that the term colony is not a stable unchanging definition and that there are in fact many different types of colonies or colonial projects. Also important to consider that it is the metropolitan/home government that is often the one making a legal distinction about how a new territory is viewed. You may find it challenging if you are looking for consistency.
One difference you are identifying is that of a settler colony vs an extractive colony. With the first, large numbers of settlers arrive, appropriate land, and seek to reestablish land, legal, and social regimes similar to where they originated. Australia, New Zealand, and the Americas are of course two of the largest examples of settler colonies, but out in the Pacific we could also add Hawai’i, New Caledonia, Guam, and West Papua. The second type of colony usually features an outside elite ruling through the apparatus of a colonial state who have no/little intention of settling in an area and whose goal is only the extraction of resources/value. In the case of European colonies, these were often in environments that Europeans found difficult to live due to climate or disease.
In the case of the French Empire, it’s Pacific claims were administered by the Ministry of the Marine for the first half the 19th century and then the Ministry of Colonies. So they existed on a separate government chain of command than a normal French department/region. Laws passed in Metropolitan France did not immediately take effect in its colonies— the colonies operated like separate legal entities within the empire of France and it was up to French appointed governors/administrators to decide which laws were appropriate for their territories or what addendums needed to be made. The French state tended to prefer this arrangement to mediate between the spirit of its metropolitan policies, which were intended for citizens who enjoyed full rights, and colonial policies, which were directed towards subjects who enjoyed only limited rights.
/u/b1uepenguin already has this more than covered - just to add that I wrote about different types of European colonial empires (including one definition of colonies) over here, in case you'd like to read a bit more about it.
Interesting that you mentioned Algeria, because in its later years it stopped being a French colony and was considered part of Metropolitan France. The collapse of the French 4th Republic was at least partially due (and arguably mostly because of) the importance placed on Algeria as a part of the Metropole as opposed to a colony during the Algerian war of independence