I know many of the higher ups in the Nazi regime were put on trial, but what about regular servicemen?
Part I
First off, there was not a "Nazi Army" within the Third Reich. The Wehrmacht, which constituted the army, navy, and air force had specific rules and regulations in place that prevented active NSDAP membership in its ranks. The Waffen-SS was an exception to this as it often had a dual role as both a military organization and as an arm of internal state security, which was the purview of the SS. This anomalous position within the command hierarchy was no small source of friction between the conventional Heer and their Waffen-SS subordinates. The Wehrmacht initially held the position that it was an apolitical institution and in the 1930s all NSDAP members had to resign from the party as a precondition of service. This restriction was waived or ignored as the military expanded and rendered the apolitical nature of the Wehrmacht fictive. But the Wehrmacht was nominally free of direct NSDAP control. However, a good many of its recruits had been socialized inside Nazi institutions like the Hitler Jugend or the the Nationalsozialistisches Kraftfahrkorps (NSKK) and Nationalsozialistisches Fliegerkorps(NSFK) (N.S. Motor and Flying Corps, respectively), which were incredibly popular among German youth. Around 30% of junior officers and NCO had been members of the party. This is not surprising given that the officer corps recruited largely from the middle class and this was a sector of society which the Third Reich had made a great deal of inroads in recruitment. An important aspect of an officer's career were also officer assessments for "National Socialist convictions" in which a qualifying officer would judge the officer's commitment to the state's ideology. Thus while a minority of the Wehrmacht were Nazis in that they were members of the NSDAP, a good portion of of its soldiers had been socialized in various Nazi organizations and milieus.
The postwar fate of these veterans was by and large pretty much normal. Denazification procedures in all occupation zones was porous and did not try very hard to sift through the mass of Germans to winnow out true believers from fellow travelers, opportunists, and followers. The sheer volume of the task led to various shortcuts such as automatically exempting Germans on account of youth or low earnings, which had the effect of exempting a large swath of the Wehrmacht's veterans. Most denazification efforts soon focused on leaders in various organizations like the HJ, but the overburdened tribunals too often took shortcuts and the Allied military governments soon became disenchanted with the whole system.
The Allies had initially envisioned a wider system of investigation and tribunals of Germans. Influenced heavily by the German emigre writers, the American military government initially pushed for a tribunal process that would simultaneously tackle both individuals as well as their institutions they represented. The International Military Tribunal (IMT) at Nuremberg tried the surviving Nazi elite was not only a form of justice, but it was to be the prelude to a larger didactic process whereby Germans would be exposed to the wider scale of Nazi crimes. The subsequent Nuremberg trials were grouped into various agencies and earned nicknames like the "Doctors' Trial," or "Ministries' Trial" because of the professions of the defendants.
These subsequent tribunals and their counterparts in the rest of Europe largely failed at their didactic purposes. German public opinion polls in the American zone during the IMT showed that a large percentage of respondents thought the IMT was unfair, although many did not seem to mind if the former elite like Goering swung. All zones reported that while there was some initial interest in the IMT, most of it waned and many Germans felt that the process went on for far too long for verdicts that were foreordained. Resistance to the tribunal process also crept up the closer these tribunals got to agencies and institutions staffed by "regular" Germans.
The Wehrmacht tribunals do show this pushback in action. Of the defendants of the IMT, it was the military chiefs like Raeder or Keitel who received the greatest public sympathy as men bound by their duties. Such resistance became more public during the subsequent Nuremberg tribunals, especially the so-called High Command trial of 1948 that tried a number of senior officers. Although the documentation of command responsibility was pretty extensive for the accused, the prosecution had enormous difficulty pinning exact charges on the accused generals, especially for charges of crimes against peace by engaging in aggressive war and engaging in a conspiracy against peace. Many of these officers also had ready defenses ready for these charges as well. In 22 June 1945 in POW Camp No. 31, a number of general officers produced the circular "The Officer Corps of the Wehrmacht and the National Socialist State" claiming that the officer corps bore no collective guilt for the crimes of the Third Reich and
the officer corps was not involved in the policies of the Führer which led to war. The General Staff and the generals did not push for war, on the contrary within the bounds of the possible they warned and advocated restraint. The German officer corps felt itself to be one of the appointed representatives of the entire German people, ...In a way that has never experienced before in its history, the German people have been misled and cheated by its own leadership.
Existing officer and veterans networks also facilitated the defense and acted as a prominent advocate for clemency both for the Western military governments and the nascent FRG politicians. The common ground of military service also facilitated a greater reluctance to press investigations into the Wehrmacht. OKH chief Franz Halder produced his own edited war diary after the war that seemingly absolved him of planning for the invasion of the USSR. Ironically, one of the indictments leveled against Halder's OKW counterpart Jodl at the IMT was for planning the invasion of the USSR, and while Jodl was guilty of doing so, OKW was relatively late to the game compared to Halder and OKH. Halder was able to successfully position himself like other Wehrmacht generals like von Manstein and Guderian as apolitical professionals and he worked as an adviser to both U.S. Army Historical Division and Adenauer's government on military rearmament. This postwar rehabilitation of the German officer corps meant foisting all of the onus for Germany's military mistakes upon Hitler and disassociating this German group from the unsavory parts of the regime.
The emerging Cold War also put a damper on the prosecution of Wehrmacht crimes. Not only did the West believe that a rearmed West Germany would be necessary to deter communism, but that they needed the professionalism of the former officer corps. Furthermore, the recasting of service under Hitler as an apolitical defense of Germany against communism found a good deal of resonance in the early Cold War Pentagon and State Department. On a more prosaic level, the Iron Curtain also meant that the Western trials were denied access to the areas of Europe where Nazi crimes were the most prominent and clear.
The result of these incomplete processes of denazification and tribunals was that the FRG went through an amnesty fever in the early 1950s as many of the postwar tribunal sentences were commuted and there was a general rollback on restrictions for the civil service and pensions due to prior political affiliations. Most Wehrmacht soldiers got full access to pensions in 1951 and the Waffen-SS veterans received conditional pensions soon after. The FRG soon saw the resurgence of various veterans pressure groups and organizations, despite the Allied military governments' initial ban on these organizations, and they became a major force in FRG politics in the Adenauer period. These groups would periodically pressure the FRG to push for amnesties and commutations of German soldiers convicted of crimes in Western Europe with some degree of success.
The failed processes of reform and punishment though did not mean that demobilized and returning Wehrmacht soldiers were of no concern to either the military governments and the successor German states. Discourse throughout the zones often expressed worry that these returning men would have returned home as defeated and broken men, leaving a vacuum in the pater familias of the postwar family. POWs who returned from Soviet captivity were considered especially vulnerable to such demasculinization and psychological trauma. Moreover, total defeat had engendered a staunch anti-military sentiment among the wider German population. One of the problems for Adenauer's drive for German rearmament was the "ohne mich" (without me) movement in which Germans rejected the very idea of military service. Such sentiments were not limited to the FRG. Data from GDR Republikflucht shows a connection between waves of defectors and rumors that the SED was going to institute its own form of conscription. Postwar veteran memoirs like Manstein's Lost Victories thus also had a domestic component to them aside from perpetuating an exculpatory narrative of an apolitical military. By emphasizing martial genius undercut by Slavic hordes and Hitler's dilettantism, the German officers could rationalize away defeat in a manner that resurrected their reputation among a German public skeptical of both the military and militarism.