They were there before the Nazis came to power. Many left Germany when the Nazis took over.

You raise an intetesting question: why were there so many top scientists in Germany in the 1920s?