I will leave the first part to somebody more qualified to answer, as the inner workings of the Japanese government at the end of the war is a tangled skein only an expert is really qualified to present all the versions of the story.

But for the second part, why did America target civilian populations, well technically the US did not target civilians. By US doctrinal standards, all the strategic bombing done in WWII were done against legitimate targets of war; that is to say, barracks, command centres, factories producing war materiel etc., and Hiroshima and Nagasaki did contain structures that fulfilled the criteria. An attack against the old capital of Kyoto was considered, but shot down because of its cultural significance. However, it was acknowledged very early in Air Corps manuals that strategic bombing campaigns will create collateral civilian damage. Using early WWII technology, you weren't really targeting anything smaller than a city, if you expect to hit anything at all.

As the war escalated, it became common for Allied bombing to target entire cities. Theorists both pre-war and during the war hypothesized that a significant attack against the population of a belligerent would cause them to rise up against their warring government and bring about peace. There was a bit of demented logic to it; if burning a hundred thousand civilians shortened the war by a few months, you might save as many soldier's lives from not having to run the meatgrinder for those few months. Military planners of WWII were usually veterans of WWI, who might remember the butchery on the Western Front as a gruesome statistic: 2,000 men a day, 20,000 a day during offensives. Under those circumstances, even the evil of killing civilians might be the lesser of two evils.

By 1945, when the two nuclear bombs were dropped, the allies had already conducted attacks of similar magnitude against Germans and even other Japanese cities. Tokyo was persistently fire bombed from 1944 onwards, and the March 9th, 1945 raid known as Operation Meetinghouse left hundreds of thousands dead (common estimates are around 100,000 but varies wildly). When you consider the fact that the US military back then did not fully understand the ramifications of radiation exposure, the dropping of the nuclear bomb on Hiroshima wasn't even a particularly dastardly for Bomber Command. It was merely an economy of force, sending one aircraft to do what normally took hundreds or thousands, and indeed, had been done over and over with other cities.

To conclude, I would like to quote Curtis LeMay, chief of the US Army Air Corp during WWII and of the Air Force beyond, who was in charge of the strategic bombing operations:

As far as casualties were concerned I think there were more casualties in the first attack on Tokyo with incendiaries than there were with the first use of the atomic bomb on Hiroshima. The fact that it's done instantaneously, maybe that's more humane than incendiary attacks, if you can call any war act humane. I don't, particularly, so to me there wasn't much difference. A weapon is a weapon and it really doesn't make much difference how you kill a man. If you have to kill him, well, that's the evil to start with and how you do it becomes pretty secondary. I think your choice should be which weapon is the most efficient and most likely to get the whole mess over with as early as possible.

LeMay also said that if the Allies had lost he would have been tried as a war criminal. The commanders involved were caught in what one commentator called "Logical Insanity". Their actions were perfectly insane, not because they were not the correct, logical act, but because the parameters are themselves insane, and therefore what we might consider evil or unnecessary were conducted rationally.

There are two main schools of thought regarding the Japanese hesitancy to end the war, even after Hiroshima. As a background, the Imperial Privy Council was preparing for the eventuality of a negotiated surrender, in spite of hard-liners in the Imperial Japanese Army that advocated mass suicide. Once surrender became an option, the guiding principle was that Japan would commence negotiations, possibly mediated by the Soviet Union, only after winning some kind of tactical victory, to save face as well as improving its negotiating conditions. This stance was reconfirmed in June 1945 after the Potsdam Declaration called for Japan's unconditional surrender, in order to preserve the emperor's position in society.

The first school of thought holds that the Japanese government saw the first bombing as a show of strength, possibly a test, but that the general tenor of the war would continue: first bombs, then shelling, then amphibious invasion, and there was still an opportunity to make a great last stand before surrendering with honor. When the second bomb was dropped, the thinking goes, it became clear to the cabinet and the privy council that the US was not going to continue the war in the same way. They had new weapons and had secured control of the air and the sea, and they were going to exploit that to its fullest to weaken the Japanese military before resorting to invasion. A valiant last stand was no longer possible, they were going to die like dogs instead. Utter surrender was the only option. What little in the way of primary sources has been made available to the public concerning this time period tends to support this perspective, albeit in most cases vaguely.

The second school of the thought is that the atomic bombs were merely a pretext for surrender, and the actual event that triggered surrender was the Soviet Union declaring war on Japan, which came the day before Nagasaki. The Russians bore old grudges against Japan, the Soviets were reputed to a lot nastier than the Americans, and preventing Communism from taking a foothold in Japan was one of the few things the Japanese military, the privy council and the civilian government could all agree on. By surrendering solely to the United States, Japan hoped to avoid losing any of its important territory as well as forestalling Soviet presence and its division into occupation zones. The argument for this is generally less supported by documentary evidence than by appeals to reason: the rabid anti-Communism of the Japanese elite is well-documented; the pre-atomic firebombing of Tokyo in March 1945, nearly as destructive as Hiroshima or Nagasaki, didn't cause a dramatic shift in policy; American intentions to bomb mainland Japan into submission were already well-known, and waiting for a second atomic bomb wouldn't change anything; and so on.

Of course, for any good dialectic, there is a synthesis, and the most likely scenario is probably a combination of the two. The introduction of the atomic bomb and the Soviet Union entering the war were like a one-two punch that convinced Japan to throw in the towel; the addition of the USSR as a belligerent was a threat, but also eliminated the possibility of a negotiated peace, and that, plus the possibility of the air war entering a newer and even deadlier phase sapped the Japanese government's will to fight.

Source: Hirohito and the Making of Modern Japan by Herbert Bix is probably the definitive English-language treatment of the negotiations and decisions of the Imperial Privy Council and the Cabinet during the time period, based on direct research of publicly released source material. IIRC he doesn't take as strong a stance on that topic as he does on others, but there's plenty of discussion about that around the Internet.

This is a frequently asked question.

As answered by our great /u/restricteddata.

Though lower level scientist did argue to use the bomb as a demonstration (in Tokyo bay), to the upper decision makers's primary consideration was to hit a population center for its "psychological effect", and that Hiroshima was chosen partly because "it is such a size that a large part of the city could be extensively damaged."

And Japan did surrender immediately after Hiroshima. They surrendered when the central decision makers received report from the scientist sent to Hiroshima to report what happened. The decision makers received the report on the night of August 8, to be discussed in the meeting of August 9. And on August 9 the USSR invaded Manchuria and the US bombed Nagasaki. Nagasaki was bombed purely because of weather and scheduling, as the original plan for the atomic bomb was regular, scheduled bombing. And after Nagasaki Truman immediately made it so his approval would be needed on all subsequent bombs.

And even after all this, with lots of discussion and even an attempted coup, it was not until August 15 that Japan accepted unconditional surrender.

So it's not actually unambiguously clear whether the Hiroshima bomb or the invasion was the larger factor in Japan's surrender, but Nagasaki likely played a smaller role than the other two.