Why, Zeus, of course...

The official protector of the Games was the community that hosted them, which for most of the Archaic and Classical period was the Peloponnesian city-state of Elis. This state dominated the smaller towns of the surrounding area, including Olympia, and thus held the stewardship of the Games. Nearby Sparta tried to seize control a few times, and there are some vague hints that possibly Argos may have made a claim, and in 364 BC the Eleians had to fight the Arkadians for the privilege. However, Elis mostly held on to the Games throughout the period thanks to its proximity to Olympia. They provided the officials who ran the Games and decided on issues related to the truce.

Given that Elis was only a medium-sized community and certainly unable to fight states like Sparta, Corinth or Athens by itself, it would have been difficult for the Eleians to enforce the Olympic Truce by military means. Still, they liked to talk the talk. At one point during the Peloponnesian War, Elis claimed that no less a state than Sparta itself had violated the truce by sending troops out in support of one of Elis' local enemies after the truce had been declared. Clearly abusing their authority as hosts of the Games, the Eleians fined the Spartans 2000 silver minai for their transgression. When the Spartans refused to pay, the Eleians forbade them from participating in the Games of 420 BC. However, the Eleians were absolutely terrified that the Spartans might actually go to war over their exclusion and ensure their participation by force. They proceeded to hold the games under as much of an armed guard as they could muster, reinforced by allies troops from Mantineia, Argos and Athens.

Thankfully for them, the Spartans decided not to invade at this time (they held on to their grudge and went to war with Elis two decades later). The first line of this answer is only partly a joke; since the Games were held in honour of Zeus, most Greeks would not think lightly of violating the truce and incurring the wrath of the god. In more cynical terms, individual states probably would not like to set a precedent here, given that truces like these offered safety and a moral high ground for vulnerable communities against aggressive neighbours.

I say "truces like these" because the Olympic Games were not the only occasion for a sacred truce in Classical Greece. The sacred festivals at Nemea, Isthmia and Delphi and the Eleusinian Mysteries also protected their participants and spectators with such an epekheiria, literally a "hands-off". In addition, individual states could plead the need to celebrate a sacred festival or observe a sacred month in order to escape imminent invasion. As long as a truce was declared and agreed upon by both sides, it would offer immunity.

You can probably see the trouble here: if one side couldn't care less about the truce and decided not to observe it, there was usually nothing that could be done about it except fearsome tutting and shaking of heads once it was already too late. There are numerous examples in Archaic and Classical Greek history of attackers taking advantage of a festival to attack their enemies while they were unarmed and preoccupied. Aineias the Tactician (4th century BC) actually recommends this as a great way to catch your enemy unprepared. While the Spartan example above is the only probable case of an actual violation of the Olympic Truce - probably because it was a festival all Greeks simultaneously participated in, and would not want to interrupt - other events might not offer as much immunity as advertised. Festivals often meant that a city-state's population was conveniently gathered outside of the walls, facilitating coups in particular.

The only religious institution that had a more regulated and active system to enforce its inviolability was the Oracle at Delphi. An alliance known as the Delphic Amphictyony, consisting of most of the major city-states of Greece, was sworn to protect it from the control of any political power. This alliance probably dates to the early Archaic period, and fought "sacred" wars several times to prevent Delphi falling into the hands of local powers who intended to use its vast reserves of dedicated riches to their own advantage. The most devastating of these was the Third Sacred War (356-346 BC), which first brought Philip II of Macedon into mainland Greece.