I know that the Manhattan Project was kept very well under wraps and that very few people actually knew what it was. My question, though, is what did the world know about the pursuit of atomic bombs before the US dropped them on Japan?
Did most countries know the US was working on it? Did they know someone was working on something but not really sure what? Or was it a complete and total shock when the bomb dropped that nobody had any idea what had just happened? Or was it something in between?
The crux of your question is "really know," which can mean a lot of things. Knowledge is not a simple yes/no binary state — it is a range of understanding. In this case, we could identify several positions of "knowledge":
That nuclear fission existed, that atomic energy could be released, that atomic bombs were hypothetically possible — this was widespread knowledge, printed in major newspapers, after 1939. Anyone caught up on science journalism would know this.
That American scientists, in particularly physicists, were working very hard on many secret wartime projects — this was also widespread knowledge, though less than the previous item. TIME magazine, for example, reported that the 1942 meeting of the National Academy of Sciences was practically empty, and that secrecy was the reason. They correctly identified MIT and University of Chicago as major centers of war research, and that World War II would be "a war of physics." ("Science Hush-Hushed," Time (5/11/1942) v. 39, no. 19, p. 92).
That the US might be working on atomic energy projects to some degree — this fact was occasionally leaked in news reports, but they were scattered (wartime censorship made it hard to prevent an initial leak, but made it easy to squelch syndication). There were many, however, who figured this out without any leaks, drawing upon the first two items here. Georgi Flerov managed to figured it out all the way over in the Soviet Union, because he realized that would be the only good explanation to the fact that literally nothing had been published by American experts on nuclear fission since the start of the war. Frequently Manhattan Project security forces found people talking about US work in this field, asked them where they had heard about it, and gotten back the answer that it was obvious this must be true.
That there was a very large project known as 'Manhattan' — this was known to many generals in the US Army, who did not know what it was building and resented it had top priority. This was known to Congressmen who attempted to audit military spending and were told to look the other direction (including Senator and future-President Truman). This was known to anyone who bothered to read through Congressional appropriations hearings (probably not many). This was discussed at cocktail hour by reporters in the Washington news scene. But very few knew what it was — the name, by itself, was all the saw, if they saw it.
That the US had a major "crash" program to develop an atomic bomb and reactors in time for use in the war — this was the real secret of the atomic bomb, the thing the Manhattan Project tried very hard to keep under wraps. The fact that atomic bombs were in theory possible, the fact that the US might have some people assigned to thinking about them — this was not so surprising. But the fact that the US had decided to go "all in" on developing a nuclear weapon with the idea that they could use it in a few years was a secret (and, in fact, the US was the only country in World War II to make this development commitment). Who knew about this? The top brass of the Manhattan Project, of course (which included the President, the Secretary of War, the heads of the branches of the armed services, the head of the project itself, the scientist-administrators who coordinated defense research, and the scientists who ran the major laboratories), and (I would estimate) a few thousand people working on the Manhattan Project itself (which is to say, about 1% of the total people working on it — ~99% of the people working on the project were not told what it was building), a handful (six) Congressmen, Vice President Wallace (but not his successor, Vice President Truman), and a scattering of other figures (e.g. AT&T's Vice President for Marketing, who helped them write the first press release after Hiroshima; a newspaper reporter for the New York Times who was "embedded" into the work; one of the Regents of the University of California, who managed contracts and insurance matters for Los Alamos).
By the middle of 1945, there were lots of rumors floating around. Journalists on the Washington beat buzzed about potential new weapons, about atomic energy. But it was not published. Some of this buzzing came from other directions — e.g., there were stories coming out of Europe about how the Allies had stopped the Germans from building some wonder weapons, and by implication one might wonder if the Allies were building their own. It was what we might call a very "messy" information scene — distinguishing truth from rumor and propaganda was difficult if you weren't on the inside of things. The Germans were bragging about all sorts of weapons — freeze bombs, rays that could stop airplane engines, atomic energy. Sorting out truth from fiction was difficult. If you go back and look with a knowing eye, you can find many references to atomic research, including many accurate ones. There was even a major leak in a Cleveland newspaper about Los Alamos, which even noted that "the Mr. Big of the city is a college professor, Dr. J. Robert Oppenheimer, called 'the Second Einstein' by the newspapers of the west coast." If you know what to look for, you can find the truth in the leaks. But one has to remember that there were swarms of such stories, some true, some nonsense. Picking out the truth would have been hard to do at the time, unless you already knew what it was.
So in some sense, there were many who knew the secret, or parts of it. In another sense, though, there is a big gap between vaguely wondering and actually knowing. There is a big gap between the "hypothetical but possible" and the "actually realized." The analogy I like to use is that of a "warp drive." I don't have to tell you what that means — you already know it implies some way of moving matter faster than the speed of light, in a way that is apparently prohibited by current laws of physics. It is a concept that has penetrated our culture through fiction and science journalism. If you are a certain flavor of geek, you will know that there are some hypothetical-and-maybe-possible ideas on the table for how you would do that (e.g. the Alcubierre drive), but that even if they did work (which isn't clear), you wouldn't expect to see a working one anytime soon. Now imagine that the President announced this morning that the US had perfected a working warp drive, had sent people to Pluto and back yesterday evening, and that an entirely new age of space exploration was in front of us. You'd be shocked — because your prior knowledge, while not zero, had not prepared you for the idea that such a thing might be rapidly realized.
This is approximately what the news of the atomic bomb was like to the American people. The idea of an "atomic bomb" was a staple of both fiction and journalism since the early 20th century. After the discovery of fission, those with a wonky bent knew that hypothetically it might be used in this way. But nobody expected one to just show up six years later, destroying entire cities in a go.
America's closest allies, the British and Canadians were well aware of the Manhattan Project and even had their own delegation to participate in and monitor development.
Even today a lot of the details of the Manhattan Project are kept hidden. The Soviet Union was well aware of the Manhattan Project and it's progress due to spies such as Morren Cohen. He even got his own stamp in the USSR for stealing atomic secrets and received several rewards.
Klaus Fuchs also stole information from the Manhattan Project and ultimately became the most famous Soviet Spy due to the sheer amount of information he sent them. It could be argued that he is the man most responsible for the USSR's development of their own atomic weapons.
Germany most likely suspected the Manhattan Project, but only because they were trying to create atomic weapons too and assumed the Americans were doing the same. I don't know if Japan suspected any atomic development.
The public was certainly shocked and surprised by the Atomic Bombs. If I remember correctly, Vice President Harry Truman wasn't even informed of the Manhattan Project until Franklin Roosevelt died and he became President. That's how closely guarded a secret it was. Even the majority of the people working on the Manhattan Project didn't fully know what they were doing there.
It's safe to say that the only other countries who were aware of the Manhattan Project and it's progress were Canada, England, and the Soviet Union.