Not sure my question is very well worded, but to be clear: In India, the Caste system is still upheld and rigidly followed by many (i.e. it is frowned upon to marry outside of your Caste, Untouchables face heavy discrimination etc.). Why has Indian culture clung on to the Caste system, yet societies with similar systems, such as Medieval Japan & Feudal Europe have basically abandoned the concept?
I can't speak for Medieval Japan (although I suspect the answer is similar) but 'feudal' Europe had nothing that even closely resembled the Indian caste system. Medieval society was indeed hierarchical and stratified, but it centred around a religion that taught that every human being was equally capable of salvation - indeed, one that taught the poor were uniquely placed for easy access to paradise. Medieval aristocrats were terrible snobs, and most believed that peasants and other commoners were inherently dirtier, stupider and fouler than them, but there were no hard-and-fast, stuck in your position forever rules. There was a caste of legally unfree people in more medieval European countries, but this is much more analogous to slavery than it is to the caste system. Free peasants and other non-nobles had very little legal or religious impediment to living their lives however they wanted. Particularly in the second half of the middle ages, many of them took advantage and huge numbers of people migrated long and short distances, both to empty frontiers and cramped cities, in pursuit of a new and better life. Being on the bottom rungs of medieval society was profoundly unpleasant and difficult, but in very different way from the Indian caste system.
TL;DR: Europe couldn't abandon the concept because it never had it in the first place!
Just to quickly answer the question from the Japanese standpoint, since the European one has been very nicely answered. Japan prior to the Meiji Restoration did also have a very hierarchical society based on the Confucian ideas. Modified to have Samurai on top instead of the farmer.
In any case as has been noted by /u/Miles_Sine_Castrum, the Japanese hierarchy had very little in common with a caste system beyond both being strict hierarchies of social order. However, in addition, Japanese society has been very violently shaken twice between it's feudal times and the present. The first during the Meiji Restoration, when the Samurai class as a legal institution was outright abandoned in its aggressive westernization, and second after World War 2, when essentially the entire country had to be rebuilt from the ground up. This lead to an essentially total abandonment of the old social system.