Philip k Dick would have been a young teenager during the war
They certainly believed that the Japanese might attack at least, as seen by the network of facilities built near the coast and also inland (such as the Naval Ordnance Plant in Pocatello, Idaho) and "passive defense" camouflage ordered for military and critical civilian facilities in case of air attack.
If there was a serious notion about invasion, the leadership didn't appear to roll it down hill. I've been through the security officer's files for Puget Sound Naval Shipyard near Seattle and there's lots of things about aerial attack, but no plans for invasion.
To help bolster OP's question some time ago a thread in /r/MapPorn showed what was supposedly an American anti-war propaganda map from 1937. It shows the US divided very similarly to the situation in Philip K. Dick's book, with the West Coast under Japanese(?) domination.
If this map is real, does anyone know the story behind it?
I've never heard of any reasonable expectation of Japanese colonization of the US mainland but the internment of Japanese-Americans/expats gives a strong indication that some sort of 5th column sabotage effort was perceived to be possible.
It also bears mentioning that The Man in the High Castle's setting is supposed to be a deliberately counter-factual parallel to the real world. Rather than America colonizing Japan and Germany the reverse happened in Dick's universe. The practical considerations of the setting were much less important than the thematic value.