I am not sure if there have been similar cases of dehumanisation and mental breakdowns in a similar fashion as those suffered by those who took part in the Final Solution. However, there have been some people who have pointed at the drastic decrease of support for public executions and brutal punishments as society 'progressed', which may indicate a negative psychological effect of these forms of punishment in the population. At the beginning of Discipline and Punishment, Foucault provides two specific examples on how punishment, within less than a century, became less brutal and more private due to the increasing humanist nature of society and the public outcry against brutal forms of punishment.

As to whether or not they led to aggressive behaviours, it is hard to say as these forms of punishment were devised to serve as deterrents against crime. Because of this, it would be unlikely that it would lead to increasing levels of aggression/bloodthirst. However, if the person who was punished was someone important or with a lot of influence within a given society, the severity of his punishment might have lead to violent reprisals from those close to him (which is why brutal punishments were usually reserved for a particular set of crimes).

In case of the German soldiers who took part in the Final Solution, I'd recommend you have a quick look at Christopher Browning's Ordinary Men. Putting the whole discussion of ideological indoctrination and the German's willingness to commit atrocities aside, Browning shows the huge psychological effect the massacres had in those who took part in them.