German citizens who peaceably opposed Hitler could still be punished, and indeed executed. The White Rose is the most famous example. People who did so more privately could still be punished if the state found out.

Most Jews were killed by special units, not just by everyday soldiers. Early in the in Final Solution, Jews were shot by members of Einsatzgruppen and Ordnungspolizei; later by, local auxiliaries (Lithuanians, Ukrainians, etc.) who were often massively drunk. About half of all Jews who died in the Holocaust died by bullets, rather than in the camps. This was both inefficient and difficult for the men, as well as instantly destroying a major source of labor, which led eventually to the camps being built. Once the camps were built, they were staffed by SS-men. Famously, no one has ever found a record of a Nazi soldier being punished for not killing a Jew.

I know less about the SS-men at the camps, but we know a lot about the operations of Ordnungspolizei because of Christopher Browning's excellent Ordinary Men. It's one of the most important works of social history in the past fifty years. He found the testimony of this one unit of the police that was staffed by "ordinary men"--middle aged, mostly middle class guys who had been police in Hamburg before the War. In their first "operation" that involved shooting Jews at point blank range, the commander explicitly gave people the option of opting out, which 12 out of 500 did without ill consequence (he never explicitly gave them the option again). Over the course of the seventeen hours of killing, several others also asked to be relieved and were. No one was disciplined for it. However, over the course of later "actions", fewer people opted out, in part because they knew it was unpleasant work for almost everyone, and they knew if they opted out, it would mean more work for their comrades in arms. A few managed to secure transfer to other units (if I recall correctly, everyone who asked, but only those with a certain amount of authority asked).

But for the most part, people who openly opposed Hitler got severely punished, but those who sought to opt out of killing Jews were not punished and (generally) allowed to opt out. If you have any interest in the later, I strongly recommend Browning's book. It's written and clearly intended for both an academic and popular audience. It's chilling, but I also couldn't put it down.

Yes, peacefully opposing the Nazi regime could get you in big trouble if you acted on those feelings. Here's some examples:

• Walter Arndt: physician; executed for denouncing the Nazi regime.
• Helene Gotthold: a Jehovah's Witness; executed for refusing to fight for the Nazis.
• Willi Graf, Kurt Huber, Christoph Probst, Alexander Schmorell, and Hans and Sophie Scholl: a group of young students; executed for distributing anti-Nazi leaflets at the University of Munich.
• Alois Grimm: Jesuit priest; executed for opposing Nazi ideology.
• Georg Groscurth: a doctor; executed for hiding Jews.
• Otto and Elise Hampel: executed for writing postcards with anti-Hitler sentiments and distributing them.
• Helmuth Hübener: executed for distributing anti-fascist and anti-war leaflets.
• Hermann Lange, Eduard Müller, Johannes Prassek, and Karl Friedrich Stellbrink: three Catholic priests and an Evangelical Lutheran pastor; executed for opposing Nazi anti-Semitism and the T4 program.
• Hermann Maas: a Protestant minister who opposed anti-Semitism and helped Jews escape. After his activities came to light, he was sent to a forced labor camp in France.
• Albert Merz: a Christadelphian; executed for refusing to fight in the war.
• Max Josef Metzger: Catholic priest; executed for his pacifism.
• Albert Richter: an athlete; executed for refusing to spy for the Nazis and opposing their regime.
• Elfriede Scholz: sister of Erich Maria Remarque; executed for "undermining morale."
• Elisabeth Schumacher: an artist; executed for attempting to warn the Soviet Union about Germany's invasion and distributing anti-Nazi materials.
• Maria Terwiel: executed for helping Jews in hiding by giving them ID and ration cards and writing propaganda against the Nazi regime.
• Elisabeth von Thadden: educator and critic of Nazi policies. Executed for supposedly being involved in the July 20 plot to assassinate Hitler, but she probably was not.

There was a certain tendancy amongst Nazi leaders to think of mass killings as an "unpleasant necessity". They did not expect soldiers to happily or enthusiastically participate in such activities. They even worried about causing damages to ordinary Germans (aka good Aryans) which were expected to return to civilian live after the war. That was one of the reasons why executions were "outsourced" away from Einsatzgruppen to Lithuanians and Ukranians.