The Nazis initially buried a lot of the victims in mass graves, but the existence of identifiable human remains was troublesome from the point of view of secrecy, as well as the problems caused by Polish neighbors looting the burial grounds in the camps that had shut down. An operation called SonderAktion 1005 was set up to exhume the bodies, burn them in crematoria and pulverize the bones, and rebury the now unrecognisable remains.

This wasn't needed in Auschwitz, since the death camp was built using lessons learned from the prior camps, so the facilities came with the infamous crematoria to begin with.

Most known places where the remains of victims were dumped or buried have been incorporated into memorial sites. Most former concentration camps will have some kind of marker.

Another thing are the so-called death marches late in the war, when the inmates of KZs in the East were marched back to the Reich to evade the Red Army - many of these marches were headed to Mauthausen camp in Austria (and then on to Gusen).
These death marches claimed many victims as well. People died of exposure and exhaustion and were often shot when they couldn't continue. Others were killed by SS guards when they tried to flee during rest stops or were killed literally in the last hours of the war (like the 223 victims of the massacre of Hofamt Priel, a village on the Danube river in Austria).
Many of the victims of the last weeks of the war were buried quickly at the site of their death and most bodies were exhumed in the following years. Some were transferred to the next cemetery, but sometimes nothing is known about them and the information about their final resting place has been lost.

There are a few initiatives, some of them private, some of them public, to mark the many crime scenes of this aspect of the holocaust.