You may be interested in this older answer of mine to a rather similar question:

Von Braun knew he couldn't be in the forefront of rocket research in Germany if he didn't join all the right clubs – and he didn't care what clubs he had to join or what crimes to turn a blind eye to as long as he was allowed to play with his rockets. You may ask whether he had “a real choice”. Yes, he had the choice of not advancing in his career, he had the choice of emigrating even, as other scientists (including Einstein) had done. But he didn't care. As Tom Lehrer put it: “Once the rockets are up, who knows where they come down, that's not my department, says Wernher von Braun." Let's not forget that what he was most famous for at the end of the war was the V2 rockets that caused such death and destruction.

Let's have a look at what he did and what he knew.

He briefly joined an SS organisation as early as November 1933 while still a student. In 1937 he joined the nazi party and in 1940 he joined the SS again. All of this was probably due to certain pressures put upon him to conform if he wanted to advance. In 1944 he was briefly arrested for refusing to put his weight behind an SS takeover of the V2 programme (he didn't think they were capable enough and preferred working with the army). Conclusion: his SS involvement was not ideologically driven, but motivated by a desire to get ahead.

The V2s were built at Dora-Mittelbau, a notoriously brutal and deadly camp. Dora was a sattellite camp of Buchenwald and the Buchenwald prisoners were terrified of being sent there because of its high death toll. During the first nine months of its existence the prisoners were living in the tunnels they were digging, in dark, wet and suffocating conditions. Many rapidly fell ill, and a staggering 35 % died in those nine months. After the dormitories were built, conditions improved slightly, but still at any one time up to a fifth of the prisoners were unable to work due to illness, compounded by exhaustion and starvation. Conditions grew steadily worse again as the war neared its end. The overall death toll of Dora's twenty months' operations ended up being 66%. Two-thirds of prisoners who entered the Dora universe never left it alive.

Von Braun was aware of these conditions as he visited the camp several times. In 1976 he said in a TV interview that "the working conditions there were absolutely horrible ... it was a pretty hellish environment." Yet he also personally selected qualified workers for Dora from among the inmate population at Buchenwald.

I will leave the summing up to Michael J. Neufeld, an historian at the Smithsonian's National Air and Space Museum:

Ultimately, it is not Wernher von Braun's membership in the SS nor his involvement in slave labor that is most bothersome-in the ranks of Nazi opportunists and minor war criminals he was one of thousands, distinguished only by the high positions he held, both during and after the Third Reich. It is his technocratic amorality, his single-minded obsession with his technical dreams, that is so disturbing. If the human race is to survive its own rapidly advancing technology in the twenty-first century and beyond, scientists and engineers will have to take moral and political responsibility for their actions-something Wernher von Braun and his colleagues signally failed to do.

Sources:

Sellier, André. A History of the Dora Camp: The Untold Story of the Nazi Slave Labor Camp That Secretly Manufactured V-2 Rockets. Ivan R. Dee, 2003.

Neufeld, Michael J. "Wernher von Braun, the SS, and concentration camp labor: Questions of moral, political, and criminal responsibility." German Studies Review (2002): 57-78.