Just to comment that this is the internet, and "we" is not clearly understood. As an Egyptian (but born in Spain) I didn't have any quarrel with Iran at the time and has no desire to enter war with nobody at that time. Perhaps the question must be rephrased?

It was never really an option for the Carter administration, knowing that:

1. The Vietnam War came to an end only four years prior. Despite anti-Iranian sentiment reaching a fever pitch in the US during the hostage crisis, Americans did not want to embroil themselves in another military conflict.

2. Iran was right in the Soviet Union's backyard. The Kremlin would not have been very comfortable with American troops so close to Soviet and Afghan borders. Keep in mind that the USSR invaded Afghanistan only a month after the hostage crisis began and were fighting opposition forces backed by the US.

3. A war would have been incredibly costly. Even if you ignore the logistics of fighting Iran's military, the US would have run into hurdles even trying to move troops into the country. Consider the countries that bordered Iran at the time: the USSR, Iraq, Afghanistan, Turkey, and Pakistan. When examining which country the US could launch an invasion from, we can immediately eliminate the USSR. Iraq would have been unwilling to allow US troops into their country, given their relationship with the Soviet Union. Likewise, Afghanistan, a pro-Soviet state fighting the American-backed mujahideen, would have been a no-go. Turkey's relationship with the US was just beginning to thaw, following the end of an embargo on arms sales by the US (originally instituted as a response to Turkey's invasion of Cyprus). Moreover, the country was on the verge of a military coup. The Americans could not realistically count on Turkey as a stable base to organize their operations from. Pakistan, while an ally of the United States, was not very well situated geographically for that kind of attack. Between the Iran–Pakistan Border and Tehran are two massive deserts (Dasht-e Lut and Dasht-e Kavir), as well as a series of mountain ranges. Imagine American troops going on that grueling ~1,800 km march to the capital, while also experiencing local resistance along the way. The US could have, alternatively, opted for an amphibious invasion, with the help of their allies in the Persian Gulf; however, they would still need to overcome the same fundamental challenges they would have faced going through Pakistan (popular resistance, geography, reaching Tehran).

That said, it is important to remember that the United States did attempt to rescue the hostages during Operation Eagle Claw. It, unfortunately, proved to be a complete disaster.

If you want to learn more about the hostage crisis from the US perspective, I would recommend Taken Hostage by David Farber.