Faith. Faith is an extremely powerful tool for a pope. Most Christian monarchs were genuinely faithful, felt as if it was part of their duties to promote religion and support the Church, and believed in the legitimacy of the Pope. It's very hard to denounce or even attack someone you believe to be the messenger of your deity on earth, even if you personally disagree with them. Not to forget that your countrymen are faithful as well, and that the local clergy is very influential; in any conflict between you and the Pope, you have no guarantee that the people will side with you. This is what happened during the Guelphs and Ghibellines era (XII and XIII centuries). Some towns and states sides with the Pope, others with the Holy Roman Emperor, and it created a conflict that lasted 50 years and had consequences for centuries more. For a long time, excommunication was also a very real threat to a monarch's power. And the Pope had exclusive control over that.

Even with taking faith out of the question, medieval and Renaissance Europe was very much a case of everyone knowing each other and having a web of political alliances, and this included the Pope. Although Italy was not yet a state in itself and was divided in duchies, realms, princedoms and republics, there was still very much a sense of Italian nationality. Most popes were Italian, and most cardinals were as well. These men had (powerful) families which themselves had many alliances.

For example, Pope Alexander VI (Rodrigo Borgia) started his papacy with a tight alliance with the Sforza family, which controlled Milan. For this, he married off his daughter to a Sforza and named Ascanio Sforza Vice-Chancellor of the Church. When this fell through, Alexander VI remarried his daughter to the Aragon family of Naples, and one of his sons also married a member of that family. When this fell through again, he then allied himself to Louis XII of France through a partnership with his oldest (still living) son, Cesare. And Alexander VI was not a "popular" pope among his peers, a main reason for this being that he was a Spaniard and not Italian by birth. All this to show that no matter what, the Papacy always had allies who could fight with it. And while the Papal armies were far from being the most powerful at the time, they weren't a small force either. (Not to mention that any army wanting to attack the Vatican would have to cross Italy from either the north or the south, which had no chance of being welcomed by the people ruling over those lands. Armies have a tendency to pillage and destroy towns on their way.)

So while military action against the Pope wasn't unheard of, it was a very risky endeavour that few monarchs dared to attempt.