Because it was seen as backing down, as capitulating, as not presenting a unified front. From some perspectives it would be seen as a form of appeasement — Soviets don't like missiles in Turkey, all they have to do is appear to put them in Cuba, then they get those missiles in Turkey taken away. The Soviets in this view would manage to not actually lose anything in the deal — they would end the crisis with more than they started with. The "non-coward" view would say "stare them down and wait for them to flinch" — basically, playing chicken with them.

And what's important to remember is that the "non-coward" view is exactly what was promulgated after the Crisis ended, all of this "we were looking eye to eye and the other fellow blinked," and so on. The removal of the Jupiters from Turkey was kept secret until well afterwards. We now know that the Crisis was resolved through compromise, not just raw toughness, but this was not how Kennedy and his supporters talked about it later.

This article from The Atlantic talks a bit about how completely B.S. much of what people think about the Crisis actually is. Most of how people talked about it for decades and decades has been basically Kennedy PR lines put forward by his former staff and friends, and the reality is much more complex.

First, some background and context:

There was a strong mindset in the leaders of the U.S. and Europe in the 1950's and 1960's against "appeasement". It stems from Neville Chamberlin's approach to dealing with Hitler and the Nazi's in the 1930's, where Hitler would make provocations and territorial demands about neighboring countries, and threaten war if his demands were not met.

Chamberlain and many other British and French leaders, remembering the horrors of the First World War just two decades earlier, and with their own countries in the grips of the Great Depression and in no mood for war, strove to keep the peace in Europe in the face of Hitler's aggressions by being very accommodating in diplomatic negotiations, sometimes to an almost absurd degree as Nazi demands became more unreasonable.

Hitler of course actually wanted and expected war. The "crises" in Czechoslovakia were entirely of Nazi manufacture, and his goals weren't just to take a slice of land mostly occupied by ethnic Germans, whom he claimed were "oppressed" -- he wanted the complete destruction of those Eastern states and outright German occupation and annexation of their territories and populations. Thus, Hitler made increasing demands, and as each one was met by the Western powers, he would announce that his previous demands were now insufficient, and demand even more. The pattern was repeated with the German military occupation of the Rhineland and German re-armament, both in violation of the Treaty of Versailles; German annexation of Austria, German brinksmanship in the Sudetenland (part of Czechoslovakia) leading to its annexation by Germany, then in breach of promises to the West, full annexation of Czechoslovakia; then finally the attack on Poland which triggered World War II. The Nazi invasion of the Soviet Union would demonstrate what Hitler's true ambitions were all along.

So, by accommodating and appeasing Hitler in an attempt to preserve the peace, they greatly strengthened his hand politically and militarily. The irony here is that German re-armament was still incomplete at the time of the Sudenten crisis, and a combined force of French and British expeditionary troops would have posed a serious threat to Germany had war come sooner. Hitler's own Generals didn't hold out much hope they could repel a Western invasion in the early 1930's; Germany just wasn't ready to fight yet.

If Chamberlain and his allies had confronted Hitler when he began his aggressions, and achieved quick progress against Nazi Germany from the West, Hitler's regime might have fallen before the war became all-encompassing.

After the war, the consequences of not facing down Hitler during the many crises of the 1930's, when it might have been easier to shed a small amount of blood now to avoid paying a much higher cost later became clear in hindsight, an entire generation of European and American leaders were convinced that nation-state aggressions must not be allowed to go unchallenged.

It's important to keep mindful of this context for many events during the Cold War, which we turn to now in the Cuban Missile Crisis.

President John F. Kennedy's father, Joseph Kennedy Sr., was ambassador to the U.K. in the years leading up to war, and was a supporter of Chamberlain's strategy of paying a high price for peace. (A price that should be noted would be paid by other nations to the East of Germany). One of the difficulties John F. Kennedy faced as President was dealing with accusations that he was weak on aggression and had a family history of "appeasement" from his father's positions two decades earlier. The failure of Kennedy to follow the failed Bay of Pigs invasion with American combat troops only reinforced that reputation.

From the "other side of the table", the viewpoints of Soviet First Secretary Khrushchev and Cuban First Secretary Castro, they had several problems to solve. The first was as the Bay of Pigs had proven, the United States was vigorously trying to remove Castro and the communist government of Cuba, and might possibly be prepared to use direct military intervention to do so. Indeed, many American leaders were strongly encouraging the U.S. to invade Cuba as the invasion of Cuban ex-patriates failed on the beach. The problem was, so close to the U.S. mainland, the Cuban communists and their Soviet allies would be severely challenged to defeat a determined U.S. invasion. But "a handful of nukes might do the job, and far more cheaply"^1.

In another theater, the U.S. had stationed nuclear tipped Jupiter missiles in Turkey, right against the Russian border. In Soviet calculations, then, there was a double benefit to placing nuclear-tipped IRBM's in Cuba - it offered a very strong deterrent to U.S. invasion, and it provided a counterbalance to the U.S. strategic position. Russia's first generation of ICBM's were not particularly successful, unreliable and inaccurate. The Soviets with good reason perceived themselves as falling behind in the arms race. Placing missiles in Cuba must have seemed like a perfectly sound strategic move, and only fair given the American missiles in Turkey.

Soviet calculations didn't fully understand the political calculations for any American administration, at the heights of the Cold War. No U.S. President could hope to remain a viable leader in the face of determined Cold Warriors and the large, virulently anti-communist U.S. government and voting populace, particularly Kennedy, whose opponents accused him of being "soft on communism" and pointing to his failure to commit American forces to support the failed Bay of Pigs invasion. Placing missiles in Cuba was intolerable to any American administration.

Some of Kennedy's advisers were recommending an immediate pre-emptive strike, possibly nuclear, against the missile sites in Cuba. Many American leaders and strategists believed that war with the Soviet Union was inevitable, and that the American advantage in nuclear weaponry in the early 1960's would give us the upper hand, but that advantage would fade over the decades as Soviet weapons and delivery systems caught up. It's not that the hawks were uncaring of the inevitable loss of life of a war, or bloodthirsty. "Better a less costly war now that we have an advantage, than a far more devastating conflict later that might not be winnable" was the thinking. Their recommendations were not without merit.

As the crisis brewed to a head, and the U.S. and Soviet union moved to the edge of a shooting war that almost inevitably would have gone nuclear and general in all theaters, Adlai Stevenson, a career diplomat, ambassador to the U.N. and former Democratic presidential nominee, and a dove in a world of hawks, made the statement you referenced. The idea of course was perceived as "weak" and not workable, and in my opinion Stevenson was far too astute to expect that his proposal would be adopted in open committee. Nonetheless he suggested a missile swap, removing the American missiles in Turkey in exchange for removing the Soviet missiles in Cuba. Someone had to say it, and Stevenson, with no elections to run and an older statesman, was brave enough to say it.

Of course, the final disposition of the missile crisis was exactly that - a missile swap. The U.S. made a secret promise to the Soviets to remove the Jupiter missiles from Turkey in 6 months, once the crisis had cooled, and a public pledge to not invade Cuba; and the Soviets would remove the missiles from Cuba immediately. The arrangement left Khrushchev at least some degree of political cover in his government, although not much -- his power began to fade and he was replaced by Leonid Brezhnev, a far more hardline leader. The nuclear arms race would escalate throughout the decade, and American thinking on "domino theory" and "appeasement" would lead the U.S. into the Vietnam debacle. The success of the deal hinged on it not being perceived as a quid-pro-quo, which was not acceptable politically or diplomatically to the U.S. administration.

In truth, removing all the posturing and political considerations, a swap was the best possible solution to defuse the impending conflict between somewhat balanced adversaries. (The Americans in the early 1960's had a significant lead on the Soviets in nuclear weaponry and missile technology; as well as a far more powerful Navy; the Soviets had a much more powerful conventional army poised to invade Western Europe) Again with O'Neill: "Strong nations need not bargain, and weak ones dare not." Bargains are most possible between somewhat equal adversaries.

Khrushchev badly miscalculated on placing the missiles in Cuba - their discovery would be inevitable, and the American reaction would not tolerate the existence of the missiles once known. Once caught out on the ledge, however, there had to be some kind of perceived compromise to allow the Soviet premier to climb back down, and the secret promise of a swap, with a public pledge by the U.S. to not invade Cuba, was the solution. It still cost Khrushchev his power not long after. Stevenson didn't have so much to lose, in the twilight of his career and life already.

^1 Paraphrasing from "Coming Apart: An Informal History of America in the 1960's" by William O'Neill.

Additional references: The Munich Analogy - The cuban missile crisis

A review of "Adlai Stevenson and the World: The Life of Adiai E. Stevenson" that offers an abstract of his life and thought

Edit: fixed the hyperlink.