During the 1963-65 Frankfurt Auschwitz Trial, one defendant, Robert Mulka, argued that to refuse orders to kill unarmed civilians would have been 'to sign his own death warrant'. The plausibility of this claim was examined by the court which found that in all probability camp officers who refused such orders would have been 'transferred to the front and/or a punishment battalion' but that no other punishment would have been likely.

Christopher Browning, writing in 1992, stated that: ' in the past 45 years no defense attorney or defendant in any of the hundreds of postwar trials has been able to document a single case in which refusal to obey an order to kill unarmed civilians resulted in the allegedly inevitable dire punishment [execution].'

Nonetheless, John Elster, whose book is the source for most of the information I've provided above, also considers it possible that guards may have subjectively feared for their lives if they did not comply with such orders, even if in fact this was extremely unlikely.

Additionally, /u/merteil2 posted on similar topic a while ago, which you can find here (although I would have thought/'misremembered' that Browning's book referred to the police battalions rather than the Einsatzgruppen?). He cites an incident where Christian Wirth, head of the camp at Belzec, threatened to shoot a guard who refused to execute a prisoner.