The short answer is: both. Kinda.

It's a complicated topic, because both played a role.

by 1943, the Japanese government had come to realize that they were probably not going to win the war. There was just a small problem: the allies had agreed on the formula of Unconditional Surrender in late 1942, which meant that a negotiated peace was not in the cards. That didn't stop the Japanese from trying. The idea was that by making the allied advance bloody enough, they would be able to force the allies to the negotiation table and get out of the war somewhat intact - they probably wouldn't be allowed to keep their colonial empire, but far more important was keeping the Emperor as an institution (other conditions were the right to organize their own disarmament and to hold their own tribunals for war crimes). This thinking led to the bloody battles at Iwo Jima and Okinawa, while the Japanese military prepared to defend against the expected allied landings in the Japanese Home Islands.

And they were successful, to a degree. The allied intelligence estimates for the landing in Japan grew increasingly pessimistic as the allies identified more and more Japanese units that were digging in near the beaches while thousands of planes were being readied for an all-out suicide attack on the landing fleet. Certainly, there were indications that the Japanese were getting dangerously low on pretty much everything from fuel to ammunition to basic foodstuffs, but by summer of 1945 there was severe doubt in the allied high command that a landing in the Home Islands was going to succeed. They looked increasingly towards a long blockade of the Home Islands in conjunction with a bombing campaign that was, by then, eradicating entire cities through the use of incendiary bombs.

The Japanese intended for Stalin to be the mediator in their negotiations with the allies. The Soviet Union had remained neutral in the war between the allies and Japan, and the Japanese were very anxious to keep them that way. They even allowed Soviet ships to move freely with lend-lease material from the US. However, the Soviets had promised the allies that they would join the war against Japan a few weeks after the war with Germany ended.

When the Soviets attacked on August 9th, the Japanese plan to get out of the war in a negotiated peace evaporated together with the last bits of their colonial empire as Soviet troops overran Manchuria in a matter of days. But by that point, the US had already dropped two nuclear bombs on Japan, and yet the war would continue until August 15, when Japan finally announced its surrender. It would seem that it was the Soviet attack and not the American bombs that finally forced Japan to surrender - although more due to the Japanese idea of using the Soviets to negotiate a peace with the allies than due to the Soviets' military successes. There was never any real chance that the Soviets could land a significant force in the Home Islands.

But here is the interesting part: when the Japanese discussed surrender internally, the Soviet attack was key since they did not believe that the allies had enough bombs to carry out a long campaign of nuclear bombings (they were correct, as it turned out). However, when Emperor Hirohito announced the surrender to the nation, he didn't mention the Soviets attack at all, but did say this:

Moreover, the enemy has begun to employ a new and most cruel bomb, the power of which to do damage is, indeed, incalculable, taking the toll of many innocent lives. Should we continue to fight, not only would it result in an ultimate collapse and obliteration of the Japanese nation, but also it would lead to the total extinction of human civilization.

So, in closing, it was the Soviet attack that finally pushed the Japanese to accept an unconditional surrender, but it was the atomic bomb that provided the fig leaf to justify it to the people.

Source: Richard Franks: Downfall

What a horrifying statistic, I'd never read that, thank you for explaining!