For instance, I assume a simple SS Soldier out in Northern France would know what's going on with respect to concentration camps, but how far up can one go until it is beyond a reasonable doubt that the individual would have known or would need to be reckless in their ignorance with regards to the concentration camps?
Alternatively, was their a level at which the allies would just not prosecute someone? Where did prosecutorial discretion end and politics begin? (interested because prosecutorial discretion is like SUPER important in my field).
Ok, in order to come to something resembling an answer here, it is important to differentiate several things:
Knowledge of the Concentration Camps as places for real and perceived enemies of the Nazi regime was widespread and publicized. Pretty much everyone knew they existed and that they weren't a particular fun place since the regime wanted people to know and even ran press articles such as this one as early as 1933.
Similarly, the knowledge of Jews being deported to "the East" as was official Nazi speak was widespread in the German population. It is rather difficult to hide the arrest and deportation of thousands of citizens from a country.
As far as the knowledge that the Jews were systematically murdered, this ran as wide as the ministerial bureaucracy (see for example the list of state secretaries present at the Wannsee Conference including the Propaganda ministry etc.), the regional governors in the form of the Gauleiter as well as most of the occupational ministry in several European countries seeing as they were responsible for a lot of the organization.
Plausible deniability exists probably on the local level such as majors and so forth but even there, local police men were often used to accompany deportations to place like Sobibor and other extermination camps. In the end if you could plausibly deny knowledge of the killing depended not so much on how high up in the hierarchy you were but rather what field of administration you were involved in.
As far as the question of proprietorial discretion goes, in case of the allies and also the German justice system, it was always a question of politics and possibilty. Aside from the question of what evidence was ready, the question often was of how direly you were needed in post-war Germany. Many judges, doctors and even police officers were able to continue their service in post war Germany simply because they were needed and there weren't people to replace them. Aside the Reich governement and some of the most egregious examples of criminals such as Hans Frank or the commander of Auschwitz virtually anyone could not be prosecuted because they were needed.
Sources:
Edith Raim: Nazi Crimes against Jews and German Post-War Justice, 2015.
Patricia Heberer, Jürgen Matthäus (ed.): Atrocities on Trial: Historical Perspectives on the Politics of Prosecuting War Crimes, NebraskaU 2008.
Saul Friedländer: The Years of Extermination: Nazi Germany and the Jews, 1939-1945, HarperCollins, 2007.
