Define "danger."

In the strictest terms, it was. The German sank numerous merchant vessels off the Atlantic coast, and attempted to land sabotage teams. In the Pacific, the Japanese sank a few ships off the coast with submarines, but also shelled a few land installations (to little effect), attempted to start forest fires using submarine based seaplanes (again, to little effect), and did pursue a sort of bomber balloon campaign (in so many words, large balloons with a clockwork sort of thing that had been calibrated to drop a small bomb payload once the right time aloft-altitude conditions were met).

If you count Alaska, the Japanese did invade two of the islands in the Aleutian chain, and the US and Canadians were forced to go evict them.

As to actual invasion, it was unlikely. The Germans as far as I've read never really plotted a US landing, while the Japanese variously floated the idea of taking Hawaii, in so many words it was a long way from ever being actually invaded (the Japanese Navy desired Hawaii and viewed it as a worthwhile objective, the Japanese Army did not believe it was feasible with existing military obligations. The Japanese Army was probably right).