American factories were huge sprawling affairs. They were huge targets, and Lockheed's web page has a decent explanation of how they tried to camouflage their entire factory in southern California to look like an ordinary suburb from the air. Such concerns of Japanese bombing the factories lingered for a short time after the outbreak of the war, but faded pretty quickly.

So why was there little concern for most of the war of American factories being targeted by bombers? They were indeed big fat targets, but they were out of reach of both the Japanese and Germans. The Japanese never operated their carriers anywhere near the western coast of the US, as they were concerned about holding on to the territory they had conquered early in the war rather than attempting bombing raids on the US with their valuable carriers. The Germans didn't even have a carrier force.

Germany had ambitious and completely unfeasable plans for an "Amerika" bomber. Longer range rockets were also planned. Japan actually used submarine aircraft carriers and used one to attack the continental US--hitting a town in Oregon with two bombs. The Japanese also used unmanned balloons with incendiary bombs and managed to hit the US, but the damage was quite limited and only killed a pregnant woman and five children.

Basically, the US could afford to have huge factories that were vulnerable to attack because they had an ocean between themselves and each of their major opponents. Any attacks that Germany or Japan attempted were inaccurate and feeble, so the risk was well worth the reward of having centralized production facilities on a huge scale. With the oceans secured by their Navy, the US could produce war material unhindered by Axis attacks.