There were a few plans for an eventual war with Japan, as a clash over dominance of the Pacific was seen as likely between the US and Japan after WWI. Most famously there was Plan Orange and the Rainbow Plans. Plan Orange came first and envisioned the US holding out in the Philippines and Guam until the military could be mobilized, then the US would move their fleets to relieve those garrisons. After this there was to be a decisive battle against the Japanese fleet. After the inevitable American victory the US would attack the Japanese home islands and win the war. Orange was an interwar plan that was incompletely implemented (one could see the B-17's on the Philippines as a portion of Orange).

The later Rainbow plans were made in the late 1930's and covered a number of contingencies. The one that came to pass was that the US would have allies in this war against the Japanese, including the UK.

---

Without that attack, America(seemingly) had no real interest in saving Europe from Hitler's conquest

This is at odds with FDR's Lend-Lease actions that were aimed at involving the US while acknowledging that the US as a whole had no real desire to go to war. Your sentiment of not "[l]etting an empire like Hitler's take over Europe" is well intentioned, but it ignores the realities of US domestic and foreign policy in the interwar period. FDR needed domestic support for military intervention and he strained what support he had in order to supply the British. This included US ships defending merchants from German submarines, giving the British a number of old destroyers in exchange for basing rights, and ramping up US rearmament--especially in aircraft.

Why did they not see it as a threat or at least a bad thing for America?

To send your sons off to war often requires more than the realization that something happening somewhere else is a bad thing. This was especially the case after WWI.

Would it have really been a bad thing for America?

This strays too far into "what if" territory for me to be comfortable answering.

---

Does that answer your question? Followup questions from OP and others are always encouraged.

Japan's decision for imperial expansion made a military conflict with the US almost inevitable due to the US commitment to defend the Philippines.

Although the forces under MacArthur's command were relatively small considering the ongoing expansion of the US military that began in 1940, they still represented a sizable chunk of the peacetime army. While the Japanese hotly debated Operation Z (the raid on Pearl Harbor), there was no equivalent debate over whether or not the Japanese should have invade the Philippines among Japanese military planners. Although the Philippine garrison was relatively weak, leaving the United States in such a geographic strong position would have negated the entire point of the Nanshin-ron (Southern Drive): to secure the natural resources of Southeast Asia for Japanese use.

In contrast, the American strategic thinking on the Philippines was quite muddled. Rainbow Plan 5, drawn up by the Navy in November 1940, wrote off the Philippines as undefendable. But the Roosevelt administration was politically unwilling to abandon the archipelago and actually reenforced the garrison in August/September with aircraft, tanks, and heavy artillery. The hope was that MacArthur could make good on his assurances that he could successfully defend the islands and it would act as something of an unsinkable aircraft carrier interdicting Japanese lines of communication. The Navy refused to reenforce the Asiatic Fleet in a significant way. FDR backed Rainbow Plan 5 and vetoed any expansion of the Asiatic Fleet in 1941. The Navy had kicked around sending a carrier or cruiser task force to be based in either Singapore, the Dutch East Indies, or the Philippines. The only real change was that the Asiatic Fleet's submarine arm received orders in November 1941 to engage in completely unrestricted submarine warfare if hostilities were to commence.

There's a danger for historians to put too much weight into the symbolism of the Pearl Harbor attack. While Pearl certainly was a major psychological shock to the American public, it was not the only attack upon American forces by Japanese units. It is advisable to remember that American wartime propaganda helped to popularize perceptions of Japanese perfidy and shaped the historical script of a surprise attack. This is not to say that the shock of Pearl Harbor was solely the creation of propaganda (far from it), but historians ignore this element at their peril.

Sources

Miller, Edward S. War Plan Orange: The U.S. Strategy to Defeat Japan, 1897-1945. Annapolis, Md: Naval Institute Press, 1991.

Willmott, H. P. Empires in the Balance: Japanese and Allied Pacific Strategies to April 1942. Annapolis, Md: Naval Institute Press, 1982.