Absolutely. Not only is it likely, I would go so far as to suggest that it is a near certainty that there are still unknown mass graves stretching across large swaths of Central and Eastern Europe that date from the Second World War.
As a matter of fact, there have been several ... mass ... graves that have been discovered in the past decade.
According to Der Spiegel (a German news magazine very similar in scope and content to Time Magazine), in 2014, some 31,000 bodies were found that were believed to be from World War II.
It's probable given the remote locations of some of the fighting that there are enough civilians and soldiers that went "missing" that there are some sites of mass burial. I guess the question I have for you is how large of a mass grave are you thinking here? Are we talking hundreds of bodies or thousands?
For example, let's picture a small (500 or so people) village that a smallish SS unit (20-30) comes through on the way to Stalingrad. The area is isolated and of low strategic value. Unfortunately for this village this SS unit is made up of some of the worst of the worst SS groups sent East. They round up the villagers telling them that they are moving them to a "safer" location. They march them out into the woods, kill them all, loot them, and then burn it down. As the village was fairly isolated and the members of this particular until get chewed up in Stalingrad, there are few who could possible know/remember the who, what, when, and where of this liquidation of this particular village. There might not be much of a mass grave left out there in the woods, but it could potentially still be there.
The Katyn Forest Massacre comes to mind (http://www.katyn.org.au/), while on a bit larger scale than the smaller atrocity that I describe, some of the gravesites are only known because of some Soviet Record Keeping. Imagine if such records had been destroyed during/after the war as a lot of documents were and the people who committed the crime (or witnesses) were already dead.
I believe that there are many mass graves of hundreds that are simply swallowed up by time and I think it is possible for larger ones to still be out there, particularly on the former European Eastern Front and possibly in some areas in China or some isolated islands in the Pacific.