The hardest part of making the atomic bomb, for the Soviets, was not "information" (e.g., stuff you can get from spies) but resources, material things, and experience ("tacit knowledge"). They needed to find uranium ore supplies in large enough quantities for use (thousands of tons of uranium oxide, and Soviet uranium was very poor quality, so that means hundreds of thousands of tons of ore were required to be mined, and the USSR had practically no known ore sources prior to beginning its program), and they needed to design and construct and operated the facilities for the creation of plutonium and enriched uranium. These were non-trivial tasks for any nation in the 1940s, when none of this was "off the shelf" and all experience had to be acquired first-hand, but certainly for a nation like the USSR which had been entirely focused on fighting World War II prior to 1945, and had considerable rebuilding of its major cities and economy to undertake.

They also deliberately did not use the spy data as efficiently as possible. They didn't just "copy" the American bomb, they used the data as one additional datasource in their investigation, not trusting it completely, and not trusting their own scientists completely, either. It was a check and a guide on their own work, and they did not even use it to avoid "blind alleys" that the Americans had avoided — because they weren't sure the Americans were correct on these matters. (And that proved a fair judgment on several fronts.)

Lastly, 4 years is not a long time to go from an order to produce a bomb and to have it, if you look at other nations' experiences. The United States, with its immense wartime resource allocation and dozens of Nobel Prize winning scientists, managed to pull it off in 2.5 years, which is still the "world record" for an atomic bomb program. The UK, by contrast, took 5 years to get an atomic bomb (1947-1952); France took 4 years (1956-1960); China took between 4 and 8 years depending on when you start "the clock" (they decided to make a bomb in 1956; they started getting Soviet help in 1958; the Soviets withdrew by 1960 and the Chinese were forced to figure it all out again; they tested their first bomb in 1964). Later countries that attempted to do it more clandestinely and on the cheap took on the order of a decade or more to get the bomb. Israel, India, Pakistan, and South Africa each took around 10 years each. North Korea took around 30 years, in fit and starts. Even the US 2.5 years means we are skipping the 1 year of the "pilot" scale of the program, which the Soviets essentially started in 1945; if you track the US and Soviet accomplishments in a more one-to-one fashion (as opposed to when the orders were given), the Soviets do pretty favorably. (The US had their first experimental reactor running in late 1942, and a bomb by mid 1945; the Soviets had their first experimental reactor running in late 1946, and a bomb by late 1949.)

Which is just to say: 4 years is actually pretty amazing for the USSR in the 1940s! Despite the ravages of war they still are in 2nd place for "fastest nuclear program ever" (tied with France, who had the advantage of doing it a decade later when all of these things were far more familiar). They didn't beat the US record, but heck, nobody has, even countries that knew what they were doing much more than the US and So In fact, it was seen as quite "fast" at the time — in 1945, most American scientists considered 5 years to be the minimum time for the USSR to get the bomb, and after the discovery of spies it was often asserted (frequently without evidence) that the spies had "saved" them some amount of time (a year to five years) that they would have otherwise taken. (I trend towards thinking the spies didn't affect the timetable very much, again because the timetable was much more firmly set by the resource problem.) Many well-informed Americans were quite shocked at their speed; some had expected it to take a decade or more, given how poorly-run and poorly-managed Soviet industry could be.

For a very nice comparative study of the US and Soviet bomb programs, see Michael Gordin's Red Cloud at Dawn. For a detailed history of the Soviet program, David Holloway's Stalin and the Bomb is a classic. For a comparative look at many nuclear programs, Jeffrey Richelson's Spying on the Bomb does a great job of bringing a lot of intelligence data together.