Previously, this thread contained a comment contending the premise of this question. I started yesterday to write something up but couldn't finish because I had to go to sleep. As to not let this go to waste, I decided to include it in this answer.

Just some examples form the most famous West German and Austrian post-war trials and from my own research:

In the Einsatzgruppen Trial in Ulm, the harshest sentence handed out was 15 years in prison for the commander of the EK Tilsit, of the EK 3 in the USSR, and KdS in Lithuania, Hans-Joachim Böhme. A man, responsible for deaths ranging in the hundred thousands. Yet, the German court sentenced him for taking part in 3.000 murders and gave him a 15 year sentence.

In the three Auschwitz Trials of the 1960s the harshest sentences were life-long prison for Wilhelm Boger (5 murders, co-responsible for the murder of 1010 people) and for two Jewish prisoners functionaries while Franz Lucas, Auschwitz doctor, and Robert Mulka, adjutant of the camp commander were sentenced to 3,25 and 14 years in prison respectively.

Walter Dejaco, architect of the Auschwitz gas chambers, and Ernst Lerch, Adjutant of Odilo Globocnik, were both acquitted by a jury despite overwhelming evidence that they were co-responsible for the murder of millions of people.

Also, just to give a taste from my own research:

• Investing the crimes of the 64th Police Reserve Battalion (later the SS. Police Regiment 5) in Serbia during the war, the public prosecutors' office Dortmund uncovered that they were involved in a death march of Jews and Serbs, that they had taken part in several mass shootings of Serbian Jews and Civilians, killing at least 600 people, and that they were instrumental to running a concentration camp in the Serbian town of Sabac. However, the public prosecutors' office decided not initiate a trial against the members of the formation. In their own words from 1964: "No racist motives were ascertainable. Furthermore the executions were carried out in a humane fashion (using only carbines and not machine pistols)"

• In their investigation into Walter Bene, the Wehrmacht commander of the Feldkommandantur in the Serbian town of Sabac, the public prosecutors' office Tübingen found that he ordered several Jews from the local camp hanged in a public square and being left to lie in the streets for several days. Again, not trial was conducted because the office held that he was unable to differentiate between Jews and communists and thus no racial or other motive for murder was recognizable, rather he had acted within the general conformities of war.

• In the investigation of Oblt. Walther, member of the 9./IR 433 the public prosecutors' office Tübingen discovered a document, which he had written and signed on the mass execution of Jews and gypsies from 1.11.1941. In it he describes said executions in detail and writes: "It's easier to shoot the Jews than the gypsies. The Jews go to their deaths very composed while they gypsies always wail and scream." In his interview with the police, Walther said that to him it was not clear that he was shooting Jews but that he was under the impression of shooting Partisans, which he understood to be within the general rules of war. Despite overwhelming evidence to the contrary, mainly Walther's own words from 1941, the public prosecutors' office decided that this was good enough and no trial was ever held despite being responsible for mass executions lasting three days and causing the deaths of hundreds of people.

The position of the Allies has been discussed above but when focusing on the democratic successor states of the Third Reich, West-Germany and Austria, several factors play a role in the fact that post-war justice and persecution of Nazi war criminals was spotty at best. Or in the words of the historian Jospeh Wulf: "You can document yourself to death with the Germans, there can be a democratic government in Bonn – the mass murders roam free, have their houses and cultivate their flowers."

The first factor is partly more a general one than a specific German one, namely, how can the criminal law of a liberal (not in the sense of left-leaning but in the sense of following the rule of law) democracy deal with genocide? The criminal law as envisaged by most democracies is not really prepared to deal with crimes in this magnitude. Most continental European democracies follow a theory of rehabilitation when it comes to their criminal law and because they regard the state as taking away an individual's freedom as necessary but problematic, they not only impose a high threshold on such things as the rules of evidence but also on strict limitations on maximum penalties for criminals. In these maximum penalties, states and societies orient themselves along the lines of things known and familiar and genocide does not fall into these categories. Somebody murdering five people is a rare enough occurrence to warrant a maximum penalty in many cases so how to deal with someone who murdered thousands of people? In most people's moral compass, somebody murdering thousands arguably deserves a harsher sentence than somebody murdering five people but the law is often not prepared for crimes of this magnitude and thus results the disconnect we feel as observers when somebody like the above mentioned Boger is sentenced to the same sentences as somebody who arguably committed crimes of a lesser magnitude.

The next factor is (West)German law itself. Germany's criminal code is still from 1871 and defines murder as "...whoever kills a human being out of murderous intent, to satisfy sexual desires, out of greed or otherwise base motives, insidiously or cruelly, or with means dangerous to the public, or in order to commit or cover up another crime...". The problem with this paragraph is the that for an act to qualify as murder, there is a high threshold of establishing intent. The above cited decisions by the various public prosecutors' offices to not persecute Wehrmacht soldiers as murders results from this threshold on intent. The argument that these acts were not committed with a racial or otherwise base intent represents the problem in these cases. With the statue of limitations on manslaughter running out in the 1960s, for many a public prosecutors' office the only way to prosecute Nazi criminals was to indict for communally abetting murder rather than murder itself.

The third legal factor lies in the rules of evidence in front of democratic courts. In order for something to qualify as murder, as stated above, a person must have killed another human being. In the case of the so-called "desk perpetrators", i.e. the people ordering deportations, the persecution needed to establish that they actually killed someone, which was hard to do under the established rules of evidence. German courts showed a high bias in favor of prosecuting perpetrators in whose cases it could be established that they indeed had killed another human being with their own hands. This is why Einsatzgruppen members were among the most prosecuted group or why the above mentioned Boger got a higher sentence than others. In his case, it could indeed be established that he had directly killed people because of his role as the Gestapo's torturer in Auschwitz.

The final and probably most important factor is the political and social will to prosecute these crimes. In many a case the Federal Republic of Germany had a vested interest in integrating former Nazis into their society and political system rather than prosecuting them. This was in part because a lot of the political elite, especially the second row, had been part of the Nazi system and because they felt it would benefit the social peace within their societies better to leave many of these criminals alone. Popular opinion in both West Germany and Austria was on the whole on their side in this matter. Nazism had experienced massive social enthusiasm and support among the population and these people didn't disappear overnight. Because nobody likes their guilt – in the moral not the judicial sense – paraded out in the open and because the West German and Austrian governments were working hard on a new national identity divorced from the horrors of the Nazi regime, the limited prosecution of Nazi war criminals was a partly conscious, partly unconscious political and social decision.

Especially when it came to former Wehrmacht soldiers, as described in the cases above, the political establishment of the FRG and Austria bought into a discourse of the clean Wehrmacht myth which also pervaded the judicial system. Thus such decisions as "they killed only communists and no racial motive was discernible" were made even within the judiciary.

And finally, at least in Austria, even if a case was prosecuted, the authorities at one point gave up because in several cases the juries in these cases acquitted the accused despite overwhelming evidence (Germany has no juries). Such was the case with Walter Dejaco and Ernst Lerch. Public opinion in Austria in the time of these cases was on the side of Nazi perpetrators, whether because of sympathies for what they had done or because of the social desire to leave the past alone is impossible to say, but in both of these cases 12 jury members decided to let mass murders go despite overwhelming evidence, which massively frustrated the public prosecutors and thus lead to a virtual stop of these cases from the early 70s to the early 90s.

Good question, friend. I know it's best to provide many sources for an answer, but I think Richard Overy's Why the Allies Won and Adam Tooze's The Wages of Destruction have sufficiently comprehensive explanations for you.

In chapter 9, "The Moral Contest," Overy explains that the Allies pretty much indisputably fought on the side of justice, and that the Axis pretty much equally indisputably did not. The historical evidence he marshals is as obvious as it is comprehensive: The Allies were fighting a defensive war (even Stalin, whatever you wish to say about him, could claim to be defending Soviet citizens from the savagery of Hitler's Einsatzgruppen), while the Axis was conquering people who had directly started nothing against Germany, Italy, or Japan, and their citizens knew it. Overy notes that the SS worked overtime to terrorize the German populace when the war turned against them, the Japanese employed "factory police" called kempei to root out "defeatism," and the Italians were never too keen on the war in the first place and kicked Mussolini out themselves (pp. 305, 309, 300, 299). As checkered as the Allied war effort might have been, even their Axis foes had to admit, through the necessity of oppressing Axis citizens, that the Allies really did have the moral high ground.

One result of this was that the Allies worked very hard to ensure they kept that high ground, or at least kept up the image of it. This was why they tried to justify strategic bombing as having a distinct military effect (according to Tooze and Overy, it certainly did, but that's another topic, though one I'd be happy to explain), and why they tried to portray Stalin as a "good guy" despite his dictatorial repression (pp. 296-298).

When Germany was defeated, this meant the Allies felt, essentially, that they had an obligation to be merciful. After all, they had fought a "just war," and they'd look like hypocrites if they subsequently tossed away the idea of justice, even for people as evidently abhorrent as Nazis. Allow me to quote fully from Overy:

"The decision to indict the leaders of the Nazi state was taken late in the war. Up to the last months the predominant view was in favor of summary execution by military firing squad, an idea proposed by the British. To their surprise Stalin strongly opposed the suggestion, on the grounds that the Allies would be accused of being afraid to give their enemies a fair trial. Roosevelt did not reject the idea of treating German leaders harshly, even of finishing the job swiftly in 'kangaroo courts.' But his successor as President, Harold Truman, was horrified by the suggestion that a liberal state should be engaged in lawless killing. In May 1945 he insisted that war criminals should be brought before an international tribunal, to answer for their crimes at the bar of world opinion." (p. 311)

Once can certainly debate how fair those trials were; Overy says Soviet judges were, to say the least, not very impartial (Ibid). However, a response from Albert Speer can shed some light on why so many of these Nazis "got away." Speer "admitted his guilt and that of all those beguiled by the system" (311-312). The key word there is "beguiled." As Overy implies above, Nazi Germany was an incredibly repressive state, with the secret police rigorously monitoring the behavior of the citizenry and barbarism and lawlessness more or less enshrined into the culture and political system itself. In such a morally denuding environment, it was very possible to have "reasonable doubt" over whether or not any given German consciously (and therefore maliciously) supported the regime, or just "went along with it," so to speak.

As you know, "reasonable doubt" is a very high bar to clear. An obviously guilty person can escape the gallows if there's a "reasonable doubt" about his crime. But that standard is what the men and women of the free world took to be justice. For the sake of maintaining that justice, it is the standard they held for their enemies. And thus the result that many who probably deserved death did not receive it following the end of World War II.

I wish I could end there, but a historian is obligated to tell the truth, and the truth is that there's another, less laudable reason the Germans were treated so leniently. Bluntly stated, it was geopolitics. To summarize what Adam Tooze wrote in the last pages of The Wages of Destruction, Germany was so crucial to the economy of Europe as a whole, and would serve so well as a bulwark against Communist expansion, that it would not be in anyone's interest (except for the Soviet Union's) to see it permanently crippled (Tooze, 673-674). Thus, it seems very likely that more than a few death-deserving Germans escaped it because they were useful to the country's reconstruction. Executing a good number of Germans (after all, the Nazis had tried to exterminate entirely the Jews, Slavs, and Roma) might have been just, but it would have made fighting the Cold War harder.

A brief aside pertaining to the deleted comments--I'll remove this if the mods ask me to. It is emphatically not the case that the Wehrmacht was in any sense "clean." In fact, its unrestrained barbarism actually hindered its fighting capacity! As Richard Overy said on page 304, "the criminalisation of warfare produced a growing indiscipline and demoralization among German forces themselves. The German army shot 15,000 of their own number, the equivalence of a whole division...As a proportion of total mobilised manpower, these figures were higher than they were for the Red Army, 3.3% against an estimated 1.25%. Desertion or refusal to obey orders increased as the war went on, and the law of the jungle seeped into the military structure itself."

For a longer answer, see /u/commiespaceinvader 's post here:

https://www.reddit.com/r/AskHistorians/comments/3xc03h/just_how_much_of_the_wehrmacht_was_dirty/cy3cxs0/

Anyways, hope this helps!

Sources cited: Richard Overy, Why The Allies Won. London: Johnathan Cape, 1995. There are more recent editions of this book, but this is the one I have.

Adam Tooze, The Wages of Destruction: The Making and Breaking of the Nazi Economy. Viking Books, 2006.