According to his book Paper Walls, David Wyman says there was a combination of nationalism, nativism and anti-semitism in addition to an economic aspect that prevented the US from accepting more Jews.

Anti-semitism. All of those countries had a conference, the Evian Conference, where they collectively decided no one would accept any Jewish refugees. In fact, they did worse than that. The US accepted far less than their quota of Jewish refugees that they had. Many anti-semitic arguments were made about why Jews should not be accepted, as well as bland excuses, like that countries didn't have the resources to help. You can read more about the Evian conference here: http://www.ushmm.org/outreach/en/article.php?ModuleId=10007698