The only source I've ever seen that has stated this was a Time magazine article published in 2001. The author explains that he first heard mention of this from President Kennedy in July 1961, and that Kennedy told him the Soviets had brought in components through diplomatic "pouches" (which cannot be inspected by the host nation, and are not necessarily always a pouch, but sometimes they are small crates, so big enough to get in a "suitcase" nuke but not something like a MIRV warhead) so as to not alert the Americans.

Now the same source says that when he gave a lecture many years later, a former analyst for the Defense Intelligence Agency told him that this was believed to be the case by himself and many others in the intelligence community for many years. This still doesn't make the claim necessarily true though. While it's certainly plausible that the Soviets were able to assemble a smaller tactical nuclear device (which would have a small yield, only about 200 tons at most), I personally lean closer to the idea that it was likely a misinformation campaign run by Soviet Intelligence to give American politicians pause when they considered any aggressive actions against the USSR.

For one, a tactical nuclear weapon assembled in the attic of the Soviet Embassy, isn't going to be very effective at a decapitation strike (one would assume that if tensions increased with the Soviets enough to where detonating the supposed embassy bomb became an option, the President, the cabinet, and the Joint Chief's would likely be outside of D.C. or far underground). This is primarily because it's not an air burst weapon (which is how conventional nuclear weapons are designed to detonate to achieve maximum effectiveness against their targets), and as previously mentioned, the yield would be quite small. So yes, some major damage would be caused in the immediate area but it wouldn't really be enough to tip the scales in the Soviet's favor unless there was a massive intelligence or diplomatic failure on the U.S.'s part. The second, is that it would be an extremely risky move that could easily be comprised by the fact that it is located in a civilian compound, and the high potential for accidents or accidental public exposure would have massive ramifications against the Soviets, both with the United States and the international community. The risk just isn't worth the potential reward.

Still, the KGB and the rest of the Soviet vanguard were quite capable of taking risk adverse actions that would seem completely off limits to many other countries. So it's possible, though I would remain skeptical as the only two sources stating the possibility of the bomb being present in the Soviet Embassy are either through hearsay or anecdotes. So maybe one day we will indeed see revealed documents proving as such, but for now, I remain skeptical.

Source: http://content.time.com/time/magazine/article/0,9171,1001206,00.html