Was there ever a white Anglo-Saxon mafia? Or more generally a Western European Protestant type?

There were and are loads of white criminal organizations. One percenter biker gangs (the traditionalist ones only accept white members), Aryan Brotherhood, Aryan Circle, Nazi Lowriders, the Dixie mafia, and the Hammerskins are the ones I can think of off the top of my head. I'm stretching the definition of WASP a bit cause many members of those gangs are not Christian but irreligious or pagans. Still, you won't find any minorities in those groups.

If not, why did what were at the time minorities form criminal organizations unlike the Christian WASP majority in the States?

Tons of reasons, but the basic answer to your question is racism. Oppression and poverty go hand-in-hand with organized crime. Multiethnic societies that persist in discriminating against minorities will find that disgruntled members of minority groups who can't move up the ladder through legitimate means will resort to crime. When trouble happened, these minorities were reluctant to go to the police because many cops were racist. Who cares if some Jew or Irishman got stabbed at Five Points? No one is mourning his death.

It's not easy to retain a sense of justice when the people who uphold the law treat you unjustly. Why bother playing by the status quo's rules when the rules are rigged against you? Making an honest living is easy when you're always going to bed with a full stomach, but what if your daily intake of food consists of a single loaf of bread shared between four siblings and two parents? Snatching a purse or mugging some dandy who decides to slum it in the bad part of town becomes a lot more appealing if it means your stomach will stop hurting and your baby sister will stop crying.

That's how it begins, and once it starts, it's impossible to bottle it back in. Children born into these types of environments are surrounded by crime and may even be born to criminal parents, which perpetuates the vicious cycle of poverty and criminality. Even for groups who no longer face any systematic racism like the Irish, Russians, Jews, or Italians, the lure of money offered by organized crime becomes difficult to resist even for people from comfortable backgrounds.