The conventional wisdom is that, as horrible as the bombings were, they were necessary to avoid a US invasion of mainland Japan which would have led to significantly more deaths on both sides.

I never really considered a reason to doubt this, but as I read more about the history of nuclear weapons, I find myself asking some surprising questions. In Kai Bird & Martin Sherwin's American Prometheus: The Triumph and Tragedy of J. Robert Oppenheimer, there's a full chapter dedicated to where the bombings took place, and why. They refer to a series of meetings in Washington after the Trinity test where the top brass made all the pertinent strategic decisions of using the bombs. Bird & Sherwin attempt to illustrate (in their book about Oppenheimer: a fairly pacifistic and diplomatic personality) that the meetings basically fell into two camps: Oppenheimer and the Los Alamos scientists on the one side, and the military brass on the other side. Apparently Oppenheimer argued for a detonation over the sea, to demonstrate power without taking lives. The military retort that they have to show what the bomb can do to a population centre, and they have to use both to show Stalin that they have an arsenal (so they can keep him in check after the war). They feature excerpts from the diaries of high-ranking Truman aides which suggest that the Japanese Empire might have been willing to surrender even before a bombing or an invasion, as long as some conditions are met (chiefly, that the office/title of Emperor won't just disappear after reconstruction).

So I thought that was interesting, and it put the bombings in a whole new light for me. Then I read Command & Control by Eric Schlosser, and he references historical arguments like Bird & Sherwin's and basically waves his hand and says that it's just conjecture and that the bombings were not only inevitable, but necessary to facilitate a surrender.

So, my question is what the historical consensus says about this issue, beyond the basic pop history books that introduced me to these ideas.

edit: I'm also shamefully ignorant of what exactly was happening between Japan and the USSR after German surrender, and if this could have influenced the decision at all.