From what I get, soccer didn't make it big in America due to baseball and gridiron already being popular there. And I understand that in Latin America the elite emulated Europeans, and the people then emulated the elite (adopting their games). But what about Continental Europe? Wouldn't they also have their own popular sports? What made soccer supplant the niches occupied in Europe but not the ones in America?
European countries had their own sports that could be placed in the spectrum of modern day rugby and modern-day football/soccer and the football clubs that were set up in the European countries were by British sailors or merchants.
I would suggest you read Jonathan Wilson's "Inverting the Pyramid" to know more about this.