1960 in Rome was not even a real issue as Italy was considered a minor Axis nation at best, that switched sides towards the end of the war. It was more enchanting to have the Olympics in the Eternal City for the first time for most.

The Tokyo Olympics did have some rumblings from Americans because Emperor Hirohito attended. However, it has to be remembered that Japan was a"friendly" military-occupied nation by the U.S. at that time, although free of direct U.S. control after the mid-50s. Most Americans had moved on to the Cold War, Vietnam, The Beetles...so it was at most WW2 vets that bristled while most people were more fascinated by the rapid economic and cultural transformation of Japan.

I mean there.