Realistically it had been clear to both Britain and France that Germany was sizing up Poland, after Hitler reneged on his promise to not dismantle Czechoslovakia it was obvious to everyone who the next target was going to be. Knowing this the allies had issued strong statements of support for Poland before the Molotov-Ribbentrop pact was signed. The actual announcement of the pact made very little difference, both Neville Chamberlain (the British PM) and Albert Lebrun (the French President) both said that the pact did not change their pledge to stand by Poland. Chamberlain event sent a letter to Hitler saying so, the poor courier who delivered the letter was subjected to one of Hitler's famous rants. However, the pact could be seen as one of the reasons the British officially signed an alliance with Poland as opposed to simple guarantees.

Funny enough the pact caused more of a storm among the Axis than it did among the allies. Italy and Japan both protested the flagrant violation of the anti-Comintern pact. And both feared a Soviet-German alliance would exclude them from the Axis.

As for why two warring ideologies suddenly got along? Politics. Both Hitler and Stalin were far more rational than we give them credit for. Stalin especially was a big believer in Realpolitik, the pact gave him time to ready his army for the impending war with Germany, and allowed him to regain territory that was once Russian. Germany essentially ended the possibility of the Soviets interfering in the German invasion of Poland and it allow Germany to take advantage of vast economic resources of the Soviet Union. The pact was so beneficial to both parties it would have been stupid not to sign it. Both sides did have to justify the pact to some of their more hardline supporters though. The Nazi minster Alfred Rosenberg complained about the alliance and said that it was just Ribbentrop's attempt to get back at the British (they had humiliated him when he was the German ambassador to Britain) and communists world wide were angry at the only real Communist state signing an alliance with the fascists who were responsible for oppressing and executing communists.

Sources:

A World At Arms by Gerhard Weinberg

Bloodlands by Timothy Snyder

The Third Reich in Power by Richard Evans

I can answer the second half of this question. From the Soviet side, the Red Army was simply not ready for war yet. Especially after the purges of the army in the prior two years, a direct confrontation with Germany was not something Stalin was interested in. He did suggest a military alliance with the Western Allies to try to contain Germany, but he was rebuffed. Thus, in addition to a lack of military readiness, the USSR was also politically isolated, which is not what you want to have on the eve of war. So when a possibility of a deal with Germany came up, Stalin took a very pragmatic position. A deal with Germany would delay the war by at least a couple of years, it would allow the USSR to move its border westward, farther away from important cities, it would eliminate Poland as an independent state (this was important because Poland was rather anti-Soviet and could plausibly be an ally of Germany or whoever else that might have invaded the USSR if things played out differently), and lastly Germany itself might be bogged down with war in the West, which would strengthen Stalin's hand for when the USSR was finally ready to join the fray. So in short, the Molotov-Ribbentrop had very little to do with ideology, and more about what was practical at the moment.

From the Germany's side, it is more straightforward. Hitler made concrete demands for the so-called Polish corridor from Poland. Poland refused. War at that point was basically a question of time. Hitler knew that invading Poland would likely trigger the Western Allies to declare war on Germany. The pact with the USSR was essentially a guarantee that the USSR would stay out of that conflict. Germany didn't want a war on two fronts. Again, this didn't mean any kind of long term alliance with the Soviet Union. It only meant that Germany didn't want to be worried about its eastern border while it was at war with the France and Britain.