Rebuilding the Temple has been an extremely fraught issue, accelerated considerably since Israel regained control of the Mount after the Six Day War. The Israeli PM at the time, Levi Eshkol, ceded official control of the Mount & the Al Asqa Mosque to the Muslim Waqf, where it continues to this day. While many religious Jews believe this was done as a naive gesture of peace, I personally think Eshkol didn't want the Mount to become a point of contention between religious & secular Israelis.

Modern Israel is a secular, western-style state with a parliamentary government. During epochs of the two temples, however, it was a theocracy, ruled by Halacha, or Jewish law. Building a Third Temple would certainly imply that Israel has once more become a Halachic state. Needless to say, gentiles with an interest in the region—excepting those Christians who believe that rebuilding the Temple is a biblical perquisite to the Second Coming—join in preferring Israel remain secular. In fact, politically, any attempt to build a Third Temple would first require demolition of the Al Asqa Mosque, & removal of Muslims from the site altogether. You can probably imagine how incredibly provocative that would be to already volatile ME relations. In fact, the Jordanian government controls the Waqf, & in spite of Israel's treaty with Egypt predating the one she has with Jordan, Jordan is really Israel's oldest neighboring ally. In 1973 King Hussein met secretly with Golda Meir to warn her of Egypt & Syria's impending strike against Sinai & the Golan Heights. Imagine what a blow to Israeli-Jordanian relations reduction of the Mosque would constitute.

Although many religious Jews are forbidden from entering the Mount for fear of accidentally treading where the Holy of Holies stood—accessible only by purified Kohanim priests—I don't think location is the issue as much as the need for a pure red heifer, which Numbers prescribes must be sacrificed, burned & it's ashes used to purify the site. The existence of said red heifer today is more theoretical than real, though there have been several candidates.

So, there are several, virtually insurmountable barriers, political & religious, to the rebuilding of the Temple. In the past few years, religious elements have attempted to bomb the Mosque, or drive Muslim worshipers away. Others have attempted to reinstate Jewish prayer at the Mount (instead of just the Western Wall), usually rebuffed by Israeli or Islamic authorities. On the other side, recall the 2000 Al Asqa Intifada, prompted in part by Sharon's needlessly provocative visit to the Mount.

The Mount is essentially a trigger set in a region that has all the stability of a lake made of nitroglycerine. Right now, maintaining the status quo is about the best anyone, other than extremists on either side, can hope for.

Given the growth of Israel's religious population, the time may come when sentiment for it outweighs the desire for Israel to remain a secular, western-style state. So, who knows what the future will bring? But, bottom line, whatever the focus of the majority of Israelis, building a Third Temple would doubtless plunge the region into chaos.

Incidentally, I've been told by religious friends, that because the Al Asqa Mosque was built where it was built, Jews cannot pray there anymore under any circumstances. That is simply not so. Jews are not allowed to pray in any place where idolatry was performed, but Islam is not regarded as idolatrous. Furthermore statues of Jupiter, then later, I believe, the Emperor Julian (who encouraged Jews to rebuild), where kept on the site for a time, but that didn't stop two attempts to rebuild the Temple, one in the 4th century that terminated in an earthquake, the other early in the 7th century that included animal sacrifice, only to end after Persians conquered Jerusalem & a Christian mob tore down the rudimentary temple in 615. So, occupation is not a religious problem, but mainly a political one.

The other answer here is excellent, but there's even more to the story-
There has been at least one attempt by some right-wing Jews to precipitate the building of the Temple.
First of all, I will say that there are many many Jews who want the Temple to be built but believe that this will happen with the coming of the Messiah. (This is an opinion which was held by such scholars as Maimonides, who wrote that the Messiah will prove himself by, among other things, building the Temple.) They will pray for the restoration of the Temple, but for theological just as much as political reasons, will not practically do anything in order to actually construct it. Some extremist groups go so far as to protest the existence of a State of Israel at all because they don't believe that Jews should control the area until the Messiah comes (these are the chassidic Jews pictured at anti-Israel rallies, visiting Ahmadinejad, etc). Some believe that the more of the area is reclaimed by Jews, the sooner the Messiah will come- this is an attitude of many (but not all) settlers in the West Bank. There is a group called the Temple Institute which has built Temple vessels in preparation for the rebuilding of the Temple. As mentioned in the other answer, many tried/try to maintain Jewish prayer on the Temple Mount with mixed success (while others are vehemently against this).
Some extreme right-wingers, however, have gone even farther than this. One famous attempt to blow up Al Aqsa to clear space for the rebuilding of the Temple was by the Jewish Underground (or Machteret) which was part of the Gush Emunim, an influential group of settlers who settled the Gush Etzion area after the Six Day War. Aside from bombing Arab vehicles in what they saw as retaliation against terrorism and protest against Camp David, one of their main goals was the bombing of Al Aqsa, about which they had several different ideas (including stealing a plane and explosives from the Israeli Air Force and crashing it into the mountain). Of course this never occurred, and following the discovery of a plan for the group to bomb Arab buses in Jerusalem many of its leading members were imprisoned and the group disbanded, but the man behind the idea to destroy Al Aqsa, Yehuda Etzion, still works on fringe efforts to reclaim the Mount for the Temple.