After WWI, I believe the use of poison gas was considered wrong and everyone agreed not to fight in such a manner anymore. But given the strong desire to take over all of Europe/the World and the fact they were gassing millions of Jews and other prisoners it seems they would have used gas anyway. This seems especially true as the Allies were closing in on Berlin and the Germans would have become very desperate to fight back anyway possible.
Even fairly nonpersistent nerve agents like Sarin and Tabun can stay in the environment for several hours or even days in concentrations high enough to sicken a human or animal (Usually not kill). So an area struck by nerve agents in preparation for an offensive would be rendered impassable for a relatively significant period of time unless you want to have a number of men and animals participating in the attack promptly be incapacitated. So in the end any tactical benefit is outweighed by the deficits of chemical weapons.
After WW1 The Geneva Convention(s) were updated, and along with a bunch of other rules regarding prisoners, medics, etc., the usage of gases and chemicals was agreed by those who signed not to be used, as they saw the horrors of these weapons in the Great War.
Think of it this way: during the Cold War, the declaration of war between the Western and Eastern powers with a nuclear firearm was mutually assured destruction - if America shot a nuke at the Soviet Union, they would shoot one right back, and we'd all end up dead. This was a similar mentality to gases and chemicals in WW2. The Nazis knew if they used chemicals and gases, chems and gases would be used right back at them. This applied for all of the powers. However Japan did use gases on the Chinese, who they'd been at war with from since the mid 1930s.
"The decision by Hitler not to use the nerve gases, which was made in or about May of 1943, was made on the basis of several considerations. The one which we now know to have been false was the belief that the Allies also had nerve gases. Of great importance was the evident reality of great Allied air strength obviously capable of making good on the repeated public announcements that poison gas would be used in retaliation for any German use of it, either on the Eastern Front or anywhere else. An equally significant restraint on the Germans was their lack of gas masks of any sort for much of the country's civilian population." Weinberg, World at Arms, pg 558
TL;DR: Hilter was deterred by the false belief that the Allies would retaliate with nerve gases if German used them, an attack that would be devastating because the Allies had air supremacy and Germany lacked gas masks for the civilian population. While the Allies had mustard gas and phosgene gas, which the British had plans in 1940 to use this gas if Germany had successfully established an invasion beachhead, Hitler was apparantly mistaken in his belief that the Allies possessed nerve gases.
The Soviets were no strangers to gas warfare and the Bolsheviks used it quiet extensively against the Whites and other opposing forces during the civil war. Given this the Germans were well aware that the Soviets had chemical weapons stockpiles and the means to deliver them, but they were also holding back due to a kind of MAD fear.
(Speculation) If the Germans didn't have the fear of chemical retaliation, poison gas would have been immensely useful in urban warfare in clearing out whole neighborhoods in Moscow (1941) or the later sieges of Stalingrad and Leningrad. The war would have at least lasted longer if any one of those three cities had fallen to the early German operations.