It wasn't a military issue (the Soviets already had ICBMs and the capability to hit American targets), and the American statesmen admitted as such in their private (but recorded) conversations:

Robert McNamara: I’ll be quite frank. I don’t think there is a military problem here … This is a domestic, political problem.

It didn't really change the balance of power much — it was still tilted strongly toward the US side, and the Soviets already had a credible capability to take out major US cities. What it did was create a political problem for the American politicians because of the perception of increased risk

As for the moral high ground — they didn't, really. And they knew it. Kennedy admitted as much in the aforementioned taped conversations:

Kennedy: Why does he put these in there, though? ... It's just as if we suddenly began to put a major number of MRBMs in Turkey. Now that’d be goddamned dangerous, I would think.

McGeorge Bundy: Well, we did it, Mr. President.

As I've written before, this isn't Kennedy being ignorant, it's Kennedy being sarcastic. Kennedy had been trying to remove the Jupiters since he took office; they were bad weapons and bad policy. That the removal of the Jupiters and the removal of the Cuban weapons were the tit-for-tat is not coincidental.

The broader question is, though, why would the US be offended by the Soviets basing weapons offshore when the US had been deploying nuclear bombers and missiles offshore for years? The answer is just cold hard realpolitik: because they saw the US/NATO deployments as being "good" and the Soviet deployments as being "bad." The Soviets repeatedly pointed out such hypocrisies.

It is worth noting that many people, then and now, pointed out US hypocrisies in the Cold War. The idea that the US had the unambiguous moral high ground was always undercut by its support of dictators, coups, juntas, basing of weapons, testing of weapons, employing of ex-Nazis, what have you. The more prevailing leitmotif of the Cold War by the 1960s was one of gray areas as opposed to stark black and white, of the compromises one makes in the name of a greater good, etc.

It's important to consider the overall balance of power. Even factoring in Western European armies, the Soviets enjoy superiority in conventional forces. In any shooting war, the majority of Western Europe will fall before the U.S. army can hope to stop them, even with the U.S. units that are present on the continent. Because the U.S. is unwilling to deploy an equal number of conventional troops and the Western Europeans are likewise, they have smaller conventional forces but can count on U.S. nuclear arms to even the odds. Because the Soviets still lack an effective means of striking the U.S. in 1962 (apart from the missiles they deploy to Cuba), a kind of balance of power evolves around this.

The deployment of missiles to Cuba undoes all of this, because suddenly the Soviets can strike with both their armies in Western Europe and launch an equally devastating strike on the United States. It's telling that de Gaulle, no immediate fan of U.S. hegemony and somebody with whom the Americans have poor relations, immediately accepts Kennedy's line of reasoning that the missiles have to be removed from Cuba. For Western Europeans, it undermines their own security dramatically, and for the nonaligned countries of the world, the whole thing dramatically raises the likelihood of a nuclear war. Even if some of the neutrals are somewhat sympathetic to the Soviet position (or, more likely, Cuba's), they also know that they will die regardless in any nuclear exchange, and so they at least move to immediately support a negotiated settlement.