For the most part, this likely has to do with them all or mostly being in locations that are large population centers. This number also provides for relatively easy divisions with the same number of teams - NFL has 4 in each, MLB has 5, etc.

Cities/Metro areas that support all Big 4 sports teams -

Phoenix, LA/Anaheim, San Francisco/Oakland/San Jose, Denver, Miami-ish (not sure where the Panthers are located), Atlanta used to, till the Thrashers moved and became the Jets, Chicago, DC, Boston, Detroit, Minneapolis/St Paul, NYC, Ohio has all 4 sports, but no one city has all 4, Philadelphia (Pittsburgh has college basketball but no NBA team), Dallas. Toronto, in Canada, doesn't have an NFL team, but they do have a CFL one.

Other cities have some teams - football and basketball are the most popular.