Well, this is an interesting one, especially as it contains lots of ridiculously long German words.
So the term you are looking for is Wehrkraftzersetzung, which literally translates to 'decomposition of defensive power'. What a beautifully efficient language. Now, a translation that is a bit less clunky would just be 'subversion of the war effort'.
So, Wehrkraftzersetzung became a crime with the Kriegssonderstrafrechtsverordnung, the 'special military penal decree'. This preparatory military law was planned in secret by Wilhelm Keitel and approved by Hitler on 17 Aug 1938 and was published for the world to see in the Reichsgesetzblatt, the 'law paper of the Reich', on 26 Aug 1939, just days after the Molotov-Ribbentrop Pact and days before the invasion of Poland.
So, what exactly constitutes Wehrkraftzersetzung?
Well, RGBl I 1939 p. 1455 ff states as follows:
Guilty are those that
publically demand or encourage the denial of service in the German or an allied army, or somehow else try to subvert or paralyze the will of the German or an allied people to defend itself.
attempt to seduce a soldier or conscript into disobedience, resistance or violence against a superior or to desertion or to unallowed leave or somehow else subvert discipline of the German or an allied army.
try to have oneself or someone else dodge military service by self-mutilation, a deceptive deed or another way either temporarily or permanently.
Alright, so that's pretty broad. Especially article 1 just screams 'totalitarian'. Any sort of questioning of the war effort was thus no longer an option.
Okay, so what about the punishment? Those are listed even before the criminal acts themselves.
(1) Wehrkraftzersetzung is punishable by death.
(2) Cases of lesser gravity can be punished with imprisonment.
(3) Next to the death and imprisonment penalties, a requisitioning of personal wealth is possible.
(4) Making false or incomplete statements to free oneself or another from the fulfillment of conscription either in part or in full will be punished with a prison sentence.
So a potential death penalty isn't great.
So what did happen to the WW2 soldiers who did refuse service? You ask your question as a hypothetical, but it actually happened. In many cases, actually.
The largest amount of conscientious objectors came from the religious community of the Jehova's Witnesses, who have a tradition of denying military service - and not only in Germany, but also countries like Canada (see Kaplan: State and Salvation, 1989). But as you might expect, Canadian JWs still had an easier time than those in Nazi Germany.
During the 1920s, the Nazi press had slandered Jehova's Witnesses (then referred to by themselves as Bibelforscher, 'bible scholars') as a Jewish-funded cult subverting German society. The JWs' promises of a biblical end to organized civilization made them a fitting puzzle piece in the narrative of Jewish Bolshevism trying to overrun the world.
They had been long-standing thorns in the regime's side on many occasions: Many JWs refused to accept the designation of the secular 1st of May as a holiday and didn't attend its celebrations for example. On the flip side, they were one of the "asocial groups" decried by Nazi propaganda and thus received no or decreased state benefits for unemployment or old age.
While some JWs tried to cut themselves off the Jewish narrative and even denounced the Jewish communities (see Wilmersdorf Declaration of 1933), the Jehova's Witnesses generally stayed a persecuted minority all the way until World War 2, when many of them refused the call to arms. In World War 1, cases of this had been rare, but they were a far smaller community then and hadn't had to live under a government that actively antagonized them.
The JW conscripts that refused service were regularly sentenced to service in the concentration camps (they even had their own triangle color, purple (sometimes shared with 7th Day Adventists)).
In order to discourage religious conscientious objectors, the Nazis became increasingly light handed with their death penalties and issued hundreds of them in the year 1940. Other groups that denied military service were other small protestant free churches like the Seventh Day Adventists, who were lumped together with the JWs as American-imported Jewish-backed religious subverters and enemies of the state and subsequently sent into concentration camps or sentenced to death, which was not necessarily mutually exclusive.