The symbol is called Totenkopf and has a long history in German military uniforms, going back to Frederick the Great's hussars 200 years before Hitler. So, while it serves jokes well, it didn't have to be seen by the men wearing it in the 1940s any more "weird" or "obviously evil" than, say, it is for modern USN personnel to wave a flag featuring an attacking rattlesnake.