I don't know about England but I'm quite surprised about your assumption regarding the castles in France. Starting in the 11th century the global power in the kingdom was shifting to the lower classes of the nobility (barons/counts mostly) and they made sure to have castles to assert their local power. I would assume that castles in England are mostly constructed after the Norman invasion while most of the castles in France were constructed prior to that event. It may not seem important but it is. Most of these castles were motte-and-bailey castles and even if french historians know where they were made most of them were destructed by the 12th/early 13th century and some of them were changed to stone castles.

Some of these castles remain, but not all, and for one good reason. After 1453 (end of the Hundred Years-War) castles lost their purpose. By that time France was the main power in Europe and the possibility of an invasion was low. The changes brought to poliorcetics by gun powder was also a big factor in the progressive loss of interest in medieval type castles. Some of them were heavily modified by their lords as Renaissance châteaux while others were either 'upgraded' or left to fall in ruins.

The final blows that sealed the fate of most french medieval castles were Richelieu (1585-1642) and Vauban (1633-1707). The first was known for many things, one of them being the destruction of many castles belonging to the french nobility. The latter did two important things. He first revolutionnized fortifications to such an extent that even the medieval castles redesigned to sustain artillery fire became once and for all obselete. The second important thing he did is the well known 'Pré-carré de Vauban', a set of fortifications built alongside the borders of France... making castles built everywhere else de facto pointless.

At last, during the 18th century, many of the urban fortifications and castles that withstood the test of time were seen as free stone quarry and thus destructed by the ever growing urban population.

Hope that helps! Sorry for bad english ;)