Overall, practical reasons stemming from the fear of retaliation figured heavily in the Germans' choice not to employ chemical weapons in the final years and months of the European War, and moral concerns were far from the only factors playing into German thinking.
First, Ian Kershaw's Hitler: A Biography reveals that by the late stages of the war Hitler was distinctly fearful of Allied initiation of chemical warfare despite their commitment to a no-first-use policy. In a February 1945 conversation with Goebbels, Hitler exclaimed that in the event of an Allied usage of chemical weapons, he would order the mass execution of large numbers of captured British and American prisoners of war. [1] Records of Hitler's conversations with his inner circle demonstrate that by early 1945, Hitler possessed little compassion for either enemy combatants or even for German soldiers. In response to the bombing of Dresden in mid-February, Hitler had to be talked out of shooting Allied POWs en masse in retaliation. One of the aspects of this proposed war crime that appealed to him most was that the Allies might execute German POWs in revenge, motivating German soldiers to fight harder. "The reason that they [German soldiers] give in so easily in the west is simply the fault of that stupid Geneva convention which promises them good treatment as prisoners. We must scrap this idiotic convention." (Ch 27) Such callous exclamations by Hitler in the final year of war challenge our confidence that Hitler still possessed much of any attachment he had ever had to conventional "moral reasoning."
Certainly, the practical impacts of Allied retaliation using chemical warfare were easy for German leadership to imagine. Stanley Lovell, former head of the wartime Office of Strategic Services's office of Research and Development, relates in his book Of Spies and Stratagems a transcript of a conversation with Luftwaffe chief Hermann Goering during a postwar interrogation, in which Goering relates that usage of chemical weapons in defense of the Reich was avoided out of fear of retaliation. As the German army relied substantially on horse-drawn transportation, chemical reprisal by the Allied armies would significantly impede logistics, as horses could not be efficiently protected from poison gas [2].
Lovell's book is not to be taken at face value, but at any rate this purported dialogue does highlight that the German army would also suffer from the effects of chemical warfare.
Allied forces had certainly made clear that they were prepared to respond to the initiation of chemical warfare. In June of 1943, President Roosevelt made it clear to the Axis powers that the Allies would retaliate in kind to any usage of poisonous gas: "Any use of gas by any Axis power... will immediately be followed by the fullest possible retaliation upon munition centers, seaports, and other military objectives." [3] Nor was this an idle threat--throughout the war the western Allies were materially prepared for the possibility of chemical escalation. A rather notable incident highlighting this fact involved the air raid on Bari, Italy in December 1943, in which the transport ship John Harvey carrying mustard-gas bombs was destroyed by German planes, inadvertently releasing its poisonous cargo.
Finally, as the Allied advance continued into Germany proper, the tactical use of chemical weapons became increasingly impractical. Fighting on German soil, chemical warfare would naturally endanger German noncombatants. At the same time, Soviet troops also eventually overran the sarin and tabun gas production facilities themselves.
In summary, it was quite clear to the Germans that they stood to lose far more from opening Pandora's Box than they would gain, and the specter of retaliatory chemical warfare, which Hitler himself feared, was a sufficient deterrent to play a large role in discouraging the initiation of chemical warfare by Nazi Germany.
[1] Ian Kershaw Hitler: A Biography
[2] Stanley Lovell Of Spies and Strategems: Incredible Secrets of World War II Revealed By a Master Spy
[3] Albert J. Mauroni. Chemical and Biological Warfare: A Reference Handbook
Follow ups;
Did the Nazis have meaningful stockpiles of chemical weapons?
What was the allies’ planned response in the event of widespread deployment of chemical weapons?
Was covered in the final paragraph of this answer by /u/kojin
Edit:fixed url
This has been asked and answered before on this subreddit and since linking to past answers is allowable by the subreddit's rules I took the time to compile a list relevant posts:
FAQ / Megathread: The Nazis, Chemical Weapons, and the holocaust (Credit to /u/commiespaceinvader)
Why didn't Nazi Germany use chemical weapons in the final defense of Germany? (see answer by /u/SplendourFalls who compiled a list of seven similar questions)
Why weren't chemical weapons used in World War 2? (see the first answer, unfortunately no credit for the user - account has been deleted)
Why didn't the Nazis use WMD's like tabun and sarin during WW2? (see answers by /u/satanshelper and /u/EyeStache)
Why were chemical weapons not widely used in WWII? (the whole discussion thread contains interesting information)
Why is it that even though the Germans and British used chemical weapons in WW1, why didn't they use them in WW2? (especially the answer by /u/Zhumanchu)