As Nikolaus Wachsmann has pointed out in his latest book KL: A History of the Nazi Concentration Camps (New York, 2015), the German public were as early as 1934 exposed to the "open terror" inflicted upon certain victims of the regime. Early concentration camps were established ad hoc in existing penitentiaries and prisons and conditions tended to worsen when the oversight of such institutions was displaced by the SS. It is important to realize that these pre-war institutions were peppered across the German states, some of which were in the heart of Berlin (Mitte, Kreuzberg, etc). Germans who lived in the immediate vicinity of the early camps witnessed and heard abuses inside. These facilities were, out of necessity, located in urban and densely populated areas.
Wachsmann writes, "contrary to the pervasive myth of ignorance about the [concentration camps], which dominated German memory for decades after the war, the camps had lodged themselves early and deeply into the minds of the population--so much so that some ordinary Germans already started dreaming about them in 1933." (p. 63).
It would be worthwhile to make the distinction between concentration camp versus death camp, the latter of which existed later during the war to ensure the wholesale, industrial, and wanton murder of European Jewry and other supposed threats to the Nazi regime. The primary objective was mass extermination. One might very well argue that the concept of "atrocity" evolved throughout the duration of the Nazi regime. For example, Martin Gründwiedl was released from Dachau in early 1934, spending more than 10 months inside for being a communist. After his release, he wrote a 30-page account of SS crimes he witnessed while incarcerated called "Dachau Prisoners Speak Out", which incorporated other witness testimonies. 400 copies of the pamphlet was distributed by the KPD (German communist party), which outlined how some prisoners were subject to daily beatings and abuse of varying degrees. Certainly unwarranted beatings and abuse were considered atrocities to Weimar sensibilities, let alone industrialized killing.
There were also some cases in which prisoners were eventually released from SS custody in the early years of the regime. This would not be the case in the 1940s, when, in the context of total war, the Nazi regime deliberately murdered victims on an unprecedented scale. Ironically, the "Holocaust by bullets", which took place mainly in the East (Galicia, Ukraine, Poland, Belarussia) was perhaps more effective for the regime.
Like Wachsmann, I argue that German civilians were certainly aware of abuses committed by the state. The argument that Germans were unaware is a result of constructing a post-war memory that is more comforting than the realities of the 1930s and 1940s.
No, they were not. The average citizen of Nazi-Germany was not aware of the concentration camps; Hitler's government made adequate attempts to cover up their atrocities to the general population and the international stage, so that at the end of the war people - both German citizens and Allied soldiers - were shocked when they came across the death camps, or heard of what was actually happening.
For instance, the Nazi's even went as far as to create a sort of "mock up" concentration camp that was used to showcase good treatment towards the Jews (the camp was Theresienstadt). The Nazis permitted delegates from the International Red Cross and the Danish Red Cross to inspect the camp. This propaganda was ultimately used to dispel rumours of the concentration camps. Also, the Nazis largely claimed to some (perhaps also the general population) that these "camps" for the Jews were only there so the Jews would have a safe-haven from the aggressive distaste towards Jews by the general population.
It was not until after the war when camps were being liberated that people realized what was going on. If you watch accurate depictions such as Band of Brothers you'll see how alarmed the German citizens were when they forced to see the extermination camps.