Who else would have taken it? Although Kaliningrad had long been German territory, the German population had been expelled in the years following World War II, and it was resettled by Russians. It had politically been part of the Russian SFSR since the war, and all of the RSFSR's territory transferred to the new Russian Federation in the fall of the Soviet Union.

I answered this a while back as well. In short, because it houses the westernmost military base, and was effectively ethnically shifted to be nearly purely ethnic Russians post-war, it was still worth holding onto for Russia. This was also why it was incorporated into the Russian SFSR in 1945 and not Lithuania or something; at the time it wasn't an issue because they were all within the USSR, and thus had no international border to cross. It only became a problem once the republics became independent, which like I noted earlier, has parallels with Crimea today.