Like you alluded to, it was arguably social, environmental, and economic changes that spurned the development of organised sports.

Keep in mind that this was an era with full industrialization, and a developing upper-middle class. For the first time in history large numbers of people had ample free time, and were not required to tend to fields for survival, or work crazy hours (at least not the upper-middle class, which is touched on).

With evenings and weekends free, people (meaning men, though women took part, albeit in limited numbers) sought out other means to occupy their time. Of particular interest was physical activities, as many of these individuals were working decidedly un-physical jobs (banking, office work, etc). Sport especially was seen as a great outlet to re-asert their masculinity through physical exploits, in that it was organised, physically demanding, and fun.

The result was that organised sports developed into larger things. No longer just the purview of children, sports were a solid event for someone; however I will make clear that these sports were strictly amateur, as it was supposed to only be those who could afford to both take time off from work, and not be paid, meaning no labourers or working class folks. Of course this slowly changed and sports professionalised, but that is beyond the question here.

Just a note as to the original post, but you might edit it to say 'major north American sports', otherwise cricket (among other sports perhaps) would surely merit inclusion as one of the five major modern sports. Also hockey internationally generally refers to what North Americans refer to as 'field hockey'.