Prisoners of war were primarily held within the Wehrmacht's prisoner of war camp system (Kriegsgefangenenwesen der Wehrmacht), but some prisoners of war, particularly Soviet prisoners of war, did find their way into the concentration camp system. The prisoner of war camp system included several different types of camps, from collection sites near the front where prisoners were processed (Armee-Gefangenen-Sammelstelle) to transit camps (Durchgangslager) to the main camps for enlisted men (Kriegsgefangenen-Mannschaftsstammlager or Stalags) and officers (Offizierslager or Oflags). This system was under the auspices of the Wehrmacht and was separate from the concentration camp system, which was run by the SS Main Economic and Administrative Office (SS-Wirtschafts- und Verwaltungshauptamt, or SS-WVHA).

However, in the case of Soviet prisoners of war, there was movement between the two camp systems. As you might already know, the Germans pursued a policy of deliberate mistreatment of Soviet prisoners of war. Of the 5.7 million Soviet POWs captured by Germany 3.3 million (57%) died. Most of these deaths occurred in the POW camps (due to starvation and disease), in transit to the camps (due to exposure, exhaustion, or starvation), or by execution on the battlefield (German troops were ordered to immediately execute all Soviet political commissars). However, there was a process known as "weeding out" (Aussonderung) that was carried out in the prisoner of war camps that held Soviet prisoners, in which the Gestapo inspected the prisoners and identified Jews as well as other groups deemed dangerous or undesirable (Communist Party members, intellectuals, etc.). These prisoners were then sent to concentration camps, where they were usually killed immediately. Other Soviet prisoners also ended up in concentration camps. Most notably, there were about 15,000 Soviet POWs who were sent to Auschwitz; by the last roll call before liberation, only 92 were still alive. The first gassing experiment using Zyklon B (the chemical later used in the gas chambers in Auschwitz as well as another extermination camp, Majdanek) was actually conducted on a group of about 600 Soviet prisoners of war in September 1941. They were also the first prisoners at Auschwitz to receive the infamous number tattoos. Other groups of Soviet POWs were sent to labor camps (including subcamps of Buchenwald and Mauthausen) and to forced labor in Germany.

As for civilian prisoners, no, they weren't all sent to concentration camps. The Nazi regime ran a large number of prisons of varying types for common criminals, as well as labor and "labor education" camps (Arbeitserziehungslager). Only certain types of prisoners were sent to concentration camps, primarily political prisoners, Jews, and other groups deemed undesirable by the Nazis (Roma, homosexuals, Jehovah's Witnesses, etc.). Most of the common criminals sent to concentration camps were classified as repeat offenders who were not desirable inmates within the normal judicial prison system. There was also a separate prison system for Wehrmacht personnel convicted of crimes against the military code of justice, which again included a wide range of offenses and a few different types of sites (standard prisons, penal camps, and penal battalions).

Recent research has shown that the Nazi concentration camp system, its judicial prison system, and the Wehrmacht camp system were much larger and more complex than we originally believed. There were probably close to 50,000 camps, prisons, and detention sites of other varieties, and new camps are still being discovered in the archives and added to the historical record.

Sources:

Reinhard Otto and Rolf Keller, Sowjetische Kriegsgefangene im System der Konzentrationslager (New Academic Press, 2019)

The United States Holocaust Memorial Museum Encyclopedia of Camps and Ghettos, Volume IV (Indiana UP, 2022)