I'm thinking in particular about Enrico Fermi and how he did lots of fundamental research up to creating the first self-sustaining nuclear chain reaction--and then went on to work on the Manhattan Project and help build the first true nuclear bombs.
In retrospect building and deploying nuclear bombs is, at least, controversial and there are many people who would point to the Manhattan Project as a prime example of a time when scientists and engineers should have just said, "No, it would be profoundly unethical to create something so destructive, I will not work on this." From my limited understanding, anti-nuclear groups formed after WWII, but I'm particularly curious if there was any controversy or soul-searching among the physics community (who presumably had some semblance of an idea of what was possible) before the war and the bombs that were dropped on Hiroshima and Nagasaki.
Nuclear fission was discovered in late 1938/early 1939. Prior to this, while the idea of atomic bombs existed in popular culture, there was no scientifically feasible way of making such things.
The first person to realize that nuclear fission would let you create a nuclear chain reaction, and from that you might be able to make a weapon, was the Hungarian physicist Leo Szilard, who at that point was living in the United States as a refugee from the Nazis. Szilard had been thinking about nuclear chain reactions (absent a reaction that would work with them) for many years, so he was "primed" to see fission in this context. He attempted to convince many scientists, including Fermi, that this was not just an interesting bit of science, and not just something that might someday lead to the industrial exploitation of atomic energy, but that this could perhaps be an imminent threat: that it might be weaponizable on a scale of years, and that the Germans might also realize that.
He had a somewhat tough time of it, because it wasn't clear that this was really true with the knowledge they had in 1939. There were some promising signs. But there were huge unknowns, and in general the transition from "new scientific discovery" to "industrial application" usually takes a decade or more.
But he was successful in convincing several scientists that this might be worth worrying about in the context of Hitler (and convinced some of them, for awhile, to stop publishing on it — not out of ethics, but out of fear of alerting the Germans to the possibility), and convinced Einstein to try and lobby the US government for more coordination of research into these matters. This started a very long chain of events that led, ultimately, to the making of nuclear weapons and the use of them against Japan, though I would emphasize that there were significant uncertainties all the way to the very end.
In terms of the ethics, you have to view the initial interest in the context of the times. Most of those lobbying for worrying about this — Szilard, Einstein, but also Eugene Wigner, Edward Teller, Rudolf Peierls and Otto Frisch in England, and so on — were Jewish scientists who had to one degree or another a personal connection to the conditions in Germany. They also knew that the German scientists still in the country were very good. So they were motivated not out of an idea of "let's make a weapon to use," so much as "we need to beat the Nazis to this world-changing technology."
Once the bomb production project got fully under way in 1942, these people ceased to have as much influence over its policy direction, and over the course of the war the focus gradually shifted from a "beat the Nazis to it" (because they gradually realized, and conclusively knew in November 1944, that the Germans had not really pursued a bomb-development project) to a "use it to win/end the war," which meant Japan.
By the time the bombs were ready, there were a significant number of scientists (including Szilard) who were lobbying for them not to be used against cities, but they were too isolated to have much effect. Considerable soul-searching was engaged with by some of the scientists (not all), but only at the point in which it was no longer a very effectual force.
This is a challenging question to answer. The best comprehensive answer is presented in The Making of the Atomic Bomb by Richard Rhodes. Rhodes does a great job of walking through many of the early pioneers of nuclear physics. I saw a talk by David Schwartz, who wrote Fermi - the last man to know everything, which had similar thoughts
I'll break down my summary of Rhodes et. al. in two parts:
At least 1914. In 1914, Science Fiction writer H.G. Wells wrote the World Set Free, where, among other things, the idea of atomic bombs was presented. Per Rhodes, Wells knew Ernest Rutherford and other scientists and attended science lectures which influenced his ideas. Small tangent, The World Set Free postulates an interesting world where because nuclear energy makes energy so cheap, this solves de-salination, hunger, and most humanitarian projects, leading to less war. However, it has tremendous economic impacts, as most working on coal projects are now unemployed. (All of this is to say, its a nuanced hypothetical that isn't entirely "nuclear energy will save the world").
However, for a time, most of the main calculations didn't have supporting data (specific, cross sections for uranium, neutron multiplication factors, human creation or knowledge of plutonium, fission as a nuclear process, etc.
Many things happened independently that made calculations possible.
You have to consider what was discovered vs. widely available and that many of these discoveries built on one another. Also, at the time of WWII, the prestigious work was on radars (see Edwin McMillan) vs. physics per say. I would argue by 1942, most PhDs in physics with access to the relevant journals could see a possible nuclear bomb, but there was still debate on the size, weight, and yield of the device (and thus its military utility).
Sources:
I am a nuclear engineer that loves history.