If you don't mind. I'm just going to more or less copy and paste the answer I gave on this thread

Initially they were sequestered in the embassy, and only allowed out to conduct basic shopping for food and other basic goods. That was only allowed with an FBI escort.

In the short term the US government rented out a luxury resort, The Homestead Resort, in Hot Springs, Virginia. Here the Japanese diplomatic staff and their families were held in relative comfort, and were mostly treated like they were on vacation. Aside from the military personnel patrolling the edge of the resort and regular searches for radios, or any other form of communication, they mostly were allowed the run of the place. Interestingly enough, although they were allowed exercise including military calisthenics, they weren't allowed to play baseball on the grounds that it was an "all-American game, and not the privilege of alien enemies."

The European Axis diplomats were simultaneously held in other similar conditions. Eventually in late March to early April 1942, all Axis officials were consolidated into the Greenbrier Hotel in Sulpher Springs, West Virginia. All this happened while the parties negotiated for a transfer of ambassadorial staffs.

On June 18th the Japanese were loaded into the Swedish ship Gripholm. They sailed to Rio de Janeiro, picked up Japanese embassy staffs from Latin American countries and then across to Lorenço Marques, Mozambique. There on July 24th, she pulled next to the Asama Maru. While the Japanese crossed from one ship to the other near the bow, the Western diplomats from Japan crossed stern to stern in the opposite direction, completing the transfer of diplomats.

This was mostly taken from Thomas Connell's book America's Japanese Hostages.