I'm not so sure that it did have so little appeal. The First Crusade, in particular, attracted a huge number of ordinary pilgrims who committed whole-heartedly to the Crusade, and mostly wound up in early graves as a result. Even among the main crusading army there was a big mixture of nationalities drawn from aristocrats from all over Western Europe, with the major exception of England (English participation doesn't really get big until the 3rd Crusade).

The difficulty in organising Crusades, and Crusade appeals which went unanswered, were logistical, rather than based on the venture itself having little appeal. Going on Crusade was monstrously expensive and had to be entirely financed by the participants. It was a venture which would last at least two years and involved a significant chance of never coming back. No matter how appealing a concept, that's not something that everyone can commit to, no matter how much they might support it in principal.

An apt comparison might be if there was an appeal for every American to go to Syria to fight ISIS for 2 years, in defence of Western Civilization. Except you have to fund it all yourself. Fighting ISIS might be an appealing prospect to the American public, and they might very well support everyone who went, but I can't imagine the numbers signing up would be very high...