Knowledge about what was happening was in no way hard to come by. The SS security service reported in March 1942 that soldiers returning from Poland or on leave were talking openly about the slaughter of the Jews and other undesirables in camps. The Chancellery complained in a session on October 9th, 1942 that discussions about the "very harsh measures" against the Jews were were being spread by soldiers on leave. Everyone knew to some extent what was going on. Civil servants knew what was going on as well. Train drivers knew where that the Jews being loaded up on trains were being taken to death camps, clerks responsible for filing reports knew from reading dispatches. Housing officials who assigned former Jewish housing to new German families knew what was happening. The list of people who knew about the Holocaust is far bigger than the list of people who didn't know. Non-Jews could see the Jews and other undesirables being loaded onto trains and shipped away. And when the Jews were all gone, a city was declared "Jew Free", and they made sure every non-Jew knew the city was cleared.
Here is a disturbing little anecdote to show you how widespread knowledge of the holocaust was, especially among those who worked for the government.
Louise Solmitz was a German woman married to a Jewish man. her daughter, Gisela fell in love with a Belgian man and they went to get married. They were denied a marriage application because the official in charge of marriages recognized Gisela as a half breed (half German, half Jewish). When Louise went in to protest and began talking to a clerk in charge of files, she mentioned her Jewish husband. The clerk was shocked when Louise said she had a Jewish husband. When Louise asked why the clerk was shocked, he simply said "Your husband is still here?".
Source:
The Third Reich at War.
What a great question. This was essentially the struggle in post war Europe, especially on who to try in the Nuremberg trials. Details about what was happening in Poland and Eastern Europe was spread through many channels. The Allies released the Joint Declaration by Members of the United Nations in 1942. This essentially put in writing that the Nazis were committing genocide against ethnic and religious groups in their conquered territories.
I recently watched the six-part documentary Auschwitz, where they address this matter. They interview several SS officers who were stationed at Auschwitz. One was in charge of collecting valuables and currency and sending it to Berlin. He was not charged in post-war, but he knew exactly what was happening at the camp. Others admitted to shooting Jews, but never were tried for war crimes.
Many people would have known about the ghettos, which were right in the middle of cities. Many people would also have known about the transports, via rail and trucks. In the documentary Auschwitz, many people in towns like Auschwitz knew about the camps and knew about smaller test sites where Gypsies and handicapped people were killed with carbon monoxide in the back of large trucks. These sites were in the middle of towns, and people were interviewed about hearing the screams, even when the Nazis tried to hide the sound with running motorcycles outside the buildings.
The Einsatzgruppen were battalions of German soldiers sent into towns to kill undesirables. These battalions moved from town to town looking for and killing Jews, Gypsies, and ethnic Slavs. The Einsatzgruppen moved across Eastern Europe, wiping out entire populations of Jews and undesirables. There's no doubt that other German infantry would have heard of these units and their purpose.
As for the high command, nearly every SS and high ranking military officers knew about the Final Solution. Many of these men were the ones who were sentenced at Nuremberg, or who committed suicide. You can see a list of these men here.
I recently watched the six-part documentary Auschwitz, where they address this matter. They interview several SS officers who were stationed at Auschwitz. One was in charge of collecting valuables and currency and sending it to Berlin. He was not charged in post-war, but he knew exactly what was happening at the camp. Others admitted to shooting Jews, but never were tried for war crimes.
To answer your question directly, there is no real answer. Its hard to know who knew and to what extent did they know. My opinion is that everyone had a sense, but turned away and did nothing to stop it. You have to remember the Nazi regime was actually very popular in Germany, and the Nazis gave the Germans a group of people to blame for their problems. Many of the infantry men and bottom ranking soldiers claim they were simply following orders, and that in the time of war, they believed all of the anti-Semitic propaganda. In an interview, an SS officer who worked at Auschwitz said something like "I could either feed my family, or the Jews that came into the camp. I believed the Jews were the enemy." Others said they knew it was wrong, but the world they lived in gave them no other choice: like many of the prisoners, you did what you had to in order to survive.
Sources: