I don't know about Nazis, but true rank-and-file Communists took the Party line, always. That is what it meant to be a "rank-and-file Communist" — it meant you were a literal card-carrying member of the Communist Party and would adopt their positions when they told you to. And the Party line was that this was a great agreement for peace.

But the Nazi-Soviet pact was, for US Communists (and probably others) a major demarcating line. It separated out the true Stalinists from the ideological Communists. Many ideological Communists, and the less committed "fellow travelers," split off from Communist Party controlled activities at this point. Even in the McCarthyist period this was seen as a major deal — if you were a Communist before the pact and split as a result of it, this meant you were less of a "stooge" than if you had stayed with the CP after the pact.

When you were a member of the Communist Party (e.g. through CPUSA), you had to abide by Party discipline. You had to adopt Party-approved positions. It was not like being a Republican or a Democrat — it was a Party-controlled organization, and so its views always aligned with the views of the Central Committee back in the USSR.

On US Communism and its responses to the Pact, there are many good resources on this, in particular the many books by Klehr and Haynes on the history of CPUSA.