The biggest secret of the Manhattan Project was that there was a Manhattan Project. That is, the secret of the atomic bomb was, largely, that an atomic bomb could be made.

That secret was revealed to the entire world on August 6, 1945.

The Soviet Union, on the other hand, had a number of well-placed spies who gave them information about the American work from 1941 onward. Stalin, in fact, knew more about the atomic bomb for much longer than did Harry Truman, who was not told about it until after Roosevelt's death.

Separate from this, what goes into making an atomic bomb? You need to acquire lots of uranium first, you need to know a lot about how nuclear chain reactions work, and you then need to produce fissile material, either by building reactors or by enriching the uranium.

The US started its project in earnest in late 1942/early 1943. (There was earlier work, but it was more or less table-top science until the decision was made to actually do the work necessary to build a bomb.) Construction of its reactors and enrichment plants started around the same time. The first plutonium production reactor went online in fall of 1944, and the first enrichment of uranium on a large scale started around the same period. This enabled them to have, by July 1945, enough high-enriched uranium for the bomb used on Hiroshima, and enough plutonium for the bomb tested at Trinity and the bomb dropped on Nagasaki. They also had another bomb's worth of plutonium in the pipeline.

So they managed to pull off all of that work in a little under 2 years of intense effort.

What about the Soviets? During the war they acquired a lot of information about what the US was doing on its bomb project, including information on the bomb design itself from the highly-placed, highly-capable mole Klaus Fuchs. During WWII, however, they did not have the manpower or resources to work on the bomb to any real degree, other than slowly working to build their first experimental nuclear reactor and collecting the intelligence information. Immediately after Hiroshima, however, Stalin accelerated the project and put Lavrentii Beria, the NKVD chief, in charge of building an atomic bomb as soon as possible.

The Soviets had a huge advantage over the US — they knew a bomb would work and they knew which methods had worked in the US case. Neither of those were secrets; the US publicized both of those facts immediately after using the bombs. They also had some technical data on how the bombs worked, but Beria did not trust spy data by itself, which meant he had his scientists check and re-check the American data. (Only a few of the Soviet scientists knew there was spy data; most of the data was "sanitized" and made to look like it came from other Soviet scientists.) In some cases this even meant having Soviet scientists "re-invent" things that the Americans had made, without realizing it.

But this kind of information did not set the timeline of the Soviet project. Unlike the Americans, the Soviets had almost no access to raw quantities of uranium. They had no major uranium sources known in their country at the end of WWII (they were able to get some from Czechslovakia at the end of the war). So the largest effort, and the real timeline of the project, was involved trying to locate uranium sources, including the exploitation (with slave labor) of very low-grade uranium ore sources (ores that would be considered too paltry for American use).

So yes, the spy information helped, but it can be overemphasized. The hardest part about the Soviets making the bomb was getting raw material. The Soviets had excellent scientists and were able to put together a strong industrial base. It took them about 4 years of continual effort to get their first bomb together — about twice as long as it took the Americans.

So personally I don't think of it as being "so soon" after the first bomb for that reason. Even at the time (in 1945), most scientists thought it would take the USSR about 5 years to get a bomb — they were slightly faster than that, but not by much, especially since they had been doing low-level scientific work on bomb questions since the war.

It only felt "very soon" to people in 1949 because they continued to say "5 years" well after 1945, without incrementing the clock downward (e.g. 4 years in 1946, 3 years in 1947, etc.). It is artifact of US estimations, and US politics, and poor US Soviet intelligence at the time, that people were "surprised" at the speed of the Soviet bomb.

The Manhattan Project was more than just Americans invention an atomic bomb. It was the continuation of Tube Alloys, the British research into fission weapons, which had been relocated to Los Alamos in part to evade German air raids. As a result, it recruited many scientists with citizenship from nations within the allies. There were several soviet agents within the Manhattan Project, notably Klaus Fuchs among others, a naturalised British Citizen who sold atomic secrets to the Soviet Union throughout the war and thereafter. Fuchs believed that the Allies were improperly withholding secrets from an ally and sought to prevent the Allies from using the bomb against Russia.

Undoubtedly, Soviet Atomic scientists would have eventually produced an atomic bomb of their own, but access to the espionage provided them with the technical understanding to rectify some of the problems they encountered. I base my understanding on a text of great relevance to this topic: Atomic: The First War of Physics and the Secret History of the Atom Bomb 1939 -1949

Another answer that hasn't been fielded here, and is part of the total, is Germans or rather German research. Namely employed, captured and kidnapped German Scientists.

As /user/restricteddata points out here,

the largest effort, and the real timeline of the project, was involved trying to locate uranium sources, including the exploitation (with slave labor) of very low-grade uranium ore sources (ores that would be considered too paltry for American use).

This is true, but just half of the equation. The idea, the execution, they had already through espionage. The beta test, they had when the US made working bombs. They needed the fuel and once they had the ore, low grade or high grade, they still didn't have enough fuel grade uranium to compete with the west. They needed to be able to refine it, and more pressingly, at speed. 5% of U-235 is needed for nuclear fuel, 90% refined, atom bombs.

Here's a declassified CIA report that on page 5 which starts a brief and somewhat accurate timetable of the development by the Steenbeck directed group and Zippe lead team, that eventually built what today is known as the Zippe Centrifuge. Gernot Zippe and other scientists either accepted invitations post war to work for the Soviets or were kidnapped. The report mentions that as well.

So they, the Soviets, knew the Americans were figuring it out, the Germans had also had a program that the Soviets knew of, and post war they immediately began to attempt to catch up using not only espionage, such as Fuchs or Rosenberg, but they also employed or kidnapped foreign scientists, particularly Germans or Austrians such as Zippe (Who was actually from Nordböhmen, Varnsdorf) to augment their research programs.

*its (no apostrophe necessary)