When and where are you asking about?

For the Manhattan Project, only one scientist is supposed to have left the project because of moral scruples. This is Josef Rotblat, a Polish physicist who was part of the British delegation to Los Alamos. After Germany surrendered he supposedly concluded there was no longer any reason to work on the bomb and left. The reality is a little more complicated than that — he also wanted to try and find his family back in Europe and being Polish put him in an awkward situation regarding security. But anyway, he's the only one who did this.

As for after the war, plenty of scientists who worked on bombs decided they no longer wanted to work on bombs anymore. Some decided not even to work on physics anymore — many decided that biology was a safer, more interesting, less security-crazy field of work.

As for other places, other times — there have always been scientists who turned down opportunities to work on weapons, or started working on weapons but later decided they didn't want to do it any more. It is hard to know how many that might be. It varies by time and place.

It is worth pointing out that there have always also been plenty of scientists willing to do military research and nuclear weapons research.