There were different burial and memorial practices for different countries, so it may be helpful if you provide more specificity about what country's militaries your ancestors served in.

I'll assume for purposes of providing a short answer that your ancestors were members of the United States military. If they are memorialized on the wall of a military cemetery in Europe, this means their remains were not identified. They are either buried as "unknowns," quite likely within the same cemetery, or their remains were not recovered.

With respect to overseas dead, the United States did not leave remains of our soldiers, identified or unidentified, in temporary graves near the battlefield. For permanent disposition, next of kin were offered the choice of repatriation for burial in a private or military cemetery or burial in a permanent overseas military cemetery operated by the American Battlefield Monuments Commission. All ABMC cemeteries in Europe were built in "friendly" territory, effectively meaning "not in Germany."

Those whose remains were not identified were memorialized by name at an ABMC cemetery, I believe in every case on a memorial wall ordered alphabetically. If remains are subsequently identified, the next of kin are offered the choice outlined above and a rosette (asterisk) is inscribed next to the name.

Most ABMC cemeteries have a number of "unknown" headstones under which actual remains are buried.