The diplomatic staff of both nations were interned in their respective countries of residence until an exchange could be negotiated by the US and Japan's nominated third parties, Switzerland and Spain respectively. This occurred in 1942. You can read an account from Robert A. Fearey, private secretary to Ambassador Grew, of what this was like here. Conditions seem to have been relatively mild.

However, the press pack that had been in Tokyo was not so lucky; assuming they were spies, the Japanese had tortured them - including waterboarding - until they were finally released.