I don't have any data relating to the numbers of Americans who defected to the USSR during the Cold War, so hopefully another contributor will come in with some information on that, but I am somewhat confused as to your suggestion that Oswald's time in the Soviet Union wasn't made "a bigger deal". As soon as Oswald's arrest was made public the US press reported on his communist sympathies and Russian connections. Here is part of a previous answer I posted to another question I can copy/paste:
Another document, which you can view here, provides a telegram sent from the Soviet Ambassador to the United States Anatoly Dobrynin to Moscow requesting advice concerning the news that Lee Harvey Oswald, who had communist connections, had been arrested and had been reported to have spent time in the USSR. Dobrynin states that after checking, it appeared the Oswald had indeed spent time in Minsk and had a Russian wife. Dobrynin notes that the media were focusing "more and more" on Oswalds left-wing sentiments and connections. Dobrynin suggested that the Soviets may wish to cooperate with the US government in providing Oswald's correspondence with his Russian wife.
In a telegram to Moscow from Andrey Gromyko, Gromyko stated that "slanderous fabrications" were being made by the US media with regard to Oswald's connections to the Soviet Union and Cuba. Gromyko suggested that if the US government requested information on Oswald's stay in the USSR, the embassy in Washington could provide them with a report. The reply from the Central Committee in Moscow approved this suggestion.
The New York Times reported on 23rd November 1963 (the day after Kennedy's assassination) that "[i]t was a $435.71 United States Government loan that enabled Lee Harvey Oswald to return to this country 18 months ago after living two and a half years in the Soviet Union." As Robert Caro noted, almost immediately after Lyndon Johnson was back on Air Force One and had been sworn in, 'television networks announced that ... a former Marine named Oswald had been arrested, and then facts, or rather alleged facts, started to emerge about Oswald's stay in Russia, about his application for Soviet citizenship, and his links with pro-Castro groups.'^1 Even before Johnson was aboard Air Force One, he 'had begun to fear that Kennedy's assassination "might be part of a worldwide plot."'^2 Oswald's communist connections would certainly feed into that fear.
Outside of the the US media, US intelligence communities were, unsurprisingly, immediately abuzz with activity over Oswald and his Russian and Cuban connections were given great attention. Despite the fact that the US intelligence community was aware of Oswald's defection to the USSR in 1959, as well as his visits to both the Soviet and Cuban embassies in late 1963, a report from the CIA to President Johnson shortly after Oswald's murder, now lost, apparently stated that 'the CIA had no hard evidence that Oswald was an agent of Moscow or Havana -- but he might be.'^3 While the US government and its intelligence apparatus eventually concluded that Oswald had worked alone and was not part of a Soviet or Cuban conspiracy to assassinate the president, a lot of attention was paid to Oswald's defection, communist sympathies, and connections to left-wing groups.
You may also be interested in this document too, also from the Wilson Center, which outlines the Soviet approval for Oswald's request to defect to the USSR in 1959, and includes Oswald's original defection request letter.
Non-digital references: