The Little Boy (Hiroshima) bomb was considered such a sure thing that even if it did not detonate correctly, they would have expected some nuclear yield just because of the impact. It was a very simple design (just two pieces of uranium that needed to come together). They had extremely high confidence in it working, despite not having tested it.

The Fat Man (Nagasaki) bomb was much more complicated and had a much higher chance of failure. But unlike the Hiroshima bomb, it was encased in 4 tons of high explosives. The high explosives would be expected to detonate upon impact, and they would have scattered the plutonium significantly around.

Here is how J. Robert Oppenheimer summarized the odds of failure for each bomb on July 23, 1945:

The possibilities of a less than optimal performance of the Little lay are quite small and should be ignored. The possibility that the first combat plutonium Fat Man will give a less than optimal performance is about twelve percent. There is about a six percent chance that the energy release will be under five thousand tons, and about a two percent chance that it will be under one thousand tons. It should not be much less than one thousand tons unless there is an actual malfunctioning of some of the components.*

So you can see how they regarded it on the eve of the mission: totally negligible fear of Little Boy failing, about a 2% chance of Fat Man being less than 1 kiloton in yield (which is totally adequate to make recovery impossible).

In both cases, one can imagine a situation in which a significant amount of fissile material was scattered about without any nuclear yield. In such a case, could the Japanese have recovered it, filtered it out from contaminants, and re-forged it into a pit? Sure. In the case of the Little Boy bomb, this would constitute them having access to a single bomb. Even in such a situation, it should be remembered that without being able to deliver such a bomb to a target, it is not worth very much. The Japanese lacked bombers with enough carry weight to carry an atomic bomb any distance, much less the long distances necessary to threaten American bases in the Pacific theater (much less threaten US continental targets). They lacked anything comparable to the B-29 bomber, in other words, and even the B-29s could only carry these bombs with modifications made to them.

In the case of the Fat Man bomb, this would not, because the implosion design was of sufficient complexity that it could not be replicated without significant effort (it took the scientists at Los Alamos basically a solid year to create it in the first place). Having a core of plutonium would not have done that much for them by itself.

The question of whether "recovery" of a dud bomb could happen was raised very early in the project (in 1943), but by the summer of 1945 nobody talked about it. Their confidence in the bombs having enough yield to make recovery impossible or impractical grew over time. You don't need total detonation to make it hard to recover material — just a tiny explosion is enough to scatter the material widely and intermix it with fission products that would make recovery difficult.

Would there have been recourse had these bombs been duds? Probably, though I have never found any concrete discussion about contingency plans. But, for example, if you followed up a "dud" raid with a firebombing raid, you would complicate/hinder any recovery efforts. Or, in the case of Hiroshima, they could have followed it up a few days later with another atomic attack. (With Nagasaki, it would have been another week or so until another bomb was ready.)

* J. Robert Oppenheimer to Brig. Gen. Thomas Farrell and Capt. W.S. Parsons (23 July 1945), Nuclear Testing Archive, Las Vegas, NV, document NV0103571.