It's a good question, and one that comes up a lot when studying the First Crusade and its startling success. However, the first thing to note is that Gregory's crusade call didn't actually fail - it was cancelled.
Gregory VII was responding to a specific request for help, sent by emperor Michael VII following the shocking defeat of the Byzantine army at the Battle of Manzikert. In terms of actual military losses it probably wasn't that bad, but strategically it was a catastrophe. Anatolia was wide open for the Turks to invade and sack, which they duly did over the following decade. For the Byzantines themselves it was one of the worst days in their entire history, and they appealed to their western neighbours for help. In particular, they tried to patch things up with the Normans of southern Italy and their renowned leader Robert Guiscard by forming an alliance by marriage. After two years of negotiations, Guiscard said no, so the Byzantines started talking to Gregory in early 1073. By late 1074 they had a deal.
This deal was pretty straightforward. The Byzantine Empire would recognise the primacy of the Bishop of Rome and effectively become catholic if Gregory VII put together a military coalition to push back the Seljuk Turks. By September 1074, the deal was on and Gregory was sending out letters to drum up support. But the Byzantines had reopened negotiations with Guiscard and reached a breakthrough. Guiscard and emperor Micheal VII agreed an alliance in August 1074 and, given Guiscard's support, the Byzantines no longer felt it necessary to cosy up to the papacy. Gregory seems to have realised this in early January of 1075, because on the 22nd he sent out a letter saying:
"Great pain and universal sorrow obsess me. The church of the Orient is moving farther from the Catholic faith, and the devil, having killed it spiritually, causes its members to perish in the flesh by the sword of his henchmen lest at any time divine grace bring them to a better mind"
Now that the alliance with Guiscard was secure, there was no need for the Byzantines to seek the pope's aid, and no longer a reason to become catholic. In the following months, Michael VII was overthrown and Guiscard invaded the Byzantine Empire with Gregory's support to restore Michael to the throne and take land in the Balkans, so in overthrowing Michael the Byzantines made enemies of both Gregory and Guiscard. Essentially, the Byzantines had managed to anger all of their prospective allies in the west, so all support for their struggle against the Seljuk Turks was called off. Gregory made it pretty clear in his letters that his primary motivation was not actually to liberate the peoples of Anatolia, but to make the eastern Christians subordinate to the catholic church. With that no longer on the table, why bother?
Looking at the chronology, there was only a period of four to five months in which recruitment for this expedition was taking place. It was called around late August or early September 1074 and it was cancelled in January 1075.
According to contemporary sources, the recruitment campaign did actually sign up tens of thousands of people. Obviously, these figures need to be taken with a bucket of salt because A) recruiting priests almost always inflated their numbers and B) just because someone signed up doesn't mean they would have actually followed through, but it's a remarkable figure nonetheless. This is perhaps why, in 1095, Urban used Gregory's proto-crusade call as a model for his own. A big win was Duke William VIII of Aquitaine. He had fought a holy war before in Spain, and responded eagerly to Gregory's letters. However, there was a fermenting civil war in France between the bishops and King Philip I, and Gregory wanted William to back the bishops so urged him to remain in France. Even with this setback in France, Gregory wrote in letters that he was still planning to gather an army and, in a rather hubristic declaration, that he would personally lead it in battle. William of Aquitaine would have to remain in France, but he was to be substituted by Matilda of Tuscany, who had long been a military ally of the papacy. However, there were also internal problems in the Holy Roman Empire that kept German rulers at home, and might require Matilda's attention as well. So in January, he called the whole thing off.
So the question isn't so much, 'Why did it fail?' but 'Why did it have to be cancelled?' and the answer seems pretty clear. There were two reasons: