Indeed there is French influence and arguably a decent amount of Portuguese influence in Southern India thanks to their settlement on Ceylon (Sri Lanka) and maritime police actions in the Indian Ocean. The Portuguese attempted hegemonic control over all Indian Ocean commerce- their success is somewhat debated by historians- with historians in the past seeing them as very successful, especially in reducing Ottoman influence in the region, but modern historians tend to look at the Portuguese as less certain in their control of trade in the Indian Ocean.

The French first set themselves up in the region by establishing control over the Indian Ocean island of Ile de Bourbon in 1664 (now known as Réunion). They were very keen to expand their overseas Empire and trading connections to the East.

The story of the French colonial establishment in India proper begins in 1673 in Chandernagore and 1674 in Pondichéry. The French set up "factories" similar to the British East India Company. These "factories" were not places of production, but essentially armed compounds of barracks, warehouses, and offices. They were the portal through which European commerce flowed in and out of the city. It was through the factory- the fortified compound- that the French and the British exerted limited political control during the late 17th and 18th centuries.

By 1692 the French had expanded to Arkat and by 1718- despite facing a serious financial crisis back at home- had expanded to the Isle de France (now Mauritius). It was in the ensuing decades after these expansion that France created three more factory-colony outposts in Indian. In rapid succession France expanded in 1723 to Yanam, 1725 Mahe and then paused before expanding again in 1739 to Karikal and then expanded its island control in the Seychelles in 1756. That is virtually the end of French expansion in the region. There are a few outliers that take place later- but the expansion to Mayotte and Madagascar are related more to French colonial expansion in Africa than in India.

So where did the French go? No where really. The Seven Years War 1754-63 effectively saw the end of French expansion in India. Though France and her allies enjoyed some success against the British and their allies in the Indian theater of the war; ultimately France could send little help to her beleaguered merchant colonies and they fell to the British. The main colony at Pondichéry was returned to the French at the end of the war; but lost it again during hostilities surrounding the French Revolution and Napoleon, and regained it again in 1814.

Believe it or not, virtually all of those French concessions (Pondicherry, Mahe, Yanam, Karaikal and Chandernagar) remained under French control until 1954 when the French concessions were reunited with the former British concessions in the newly independent Indian nation-state.

To add to what /u/b1uepenguin has said, Portugal actually managed to retain several colonies in India even after India proper became independent. The first two colonies to be absorbed into India were Dadra and Nagar Haveli, which was occupied by pro-Indian forces in 1954. The Indian army then invaded the remaining enclaves of Daman and Diu, as well as the largest colony of Goa in 1961 as part of 'Operation Vijay'.