For my answer I'll largely be following the NFL's timeline, as they seem to be the a big focus on your question, and also paint the best picture of what happened. The NBA has a lot of the same factors, but happened a few decades later so a lot of it was more abbreviated.

It's hard to understand how different the NFL was when it started and for the next few decades. It is such a monetary and cultural juggernaut that it once being even only moderately popular is weird to conceptualize. A big part of why small town teams "worked", or at least kept being attempted, is starting a team was so much simpler back then. It's not a huge exaggeration to say you basically needed $100, 11 guys, a coach and a field. In fact, as far as I can tell that's about the only real requirement. There's a tiny blurb here about how the Green Bay Packers were formed where Curly Lambeau borrowed $500 from his employer, a packing company, for equipment and permission to use one of their fields for practice. This is nearly unfathomable to think of now that teams are some of the most valuable entities on the planet. As another example, The Portsmouth Spartans were sold in 1934 for somewhere around $15,000 and were renamed the Detroit Lions. Today, they are the 30th most valuable franchise at a measly $2.4B.

In today's sports world, TV revenue is far and away the cash cow. Obviously there were no TV's when the league started, and even once it did the money leagues make has only recently gotten insane. The league recently signed a $100B 8-year tv deal after getting $39B from 2014-2022 when they had up to that point in history made $20B from TV rights. I am not business savy enough to know exactly what factors led to this but suffice to say there is far more money involved today than ever before. Even just looking at attendance, in 1925 585,000 people went to watch an NFL game and that was the most for nearly a decade to follow. This season, which isn't even over, has already had 14 million people come to games If you actually look through the first link, you can see most teams had 5000 or fewer fans at plenty of games. The big city teams (fyi Frankford Yellowjackets were basically in Philadelphia) had high attendance but most teams were very much not so.

Another big factor is the player salaries. Until 1943 players were not allowed to re-enter the field after they had been substituted off, meaning they had to play both sides of the ball. An entire post could be dedicated to the increase in specialization of football players, but suffice to say this also contributed to why you did not need nearly as many players as you do today. Teams now can have several guys signing $100M contracts, but the average salary of an NFL player was under $6000 during the early days. That works out to around $80k in today's money, and while that's certainly not a bad salary, is less than many players make in a single game. Most players were paid on a per-game basis, and if you look back at my attendance link, you can see plenty of these early, small town teams only played a few games each year. Going back to my last link, salary averages jumped from $198k in 1986 to $800k in 1993. So even relatively recently it was far cheaper to run an NFL team.

Finally, population distribution was way different at that time. Missouri had about the same population as California in 1920. Michigan, Ohio, Pennsylvania, New York, Illinois, and Massachusetts all had more. Plenty of other Midwestern states were over 2 million (California had about 3.5). While trains were quite good at moving people by this point, there is no comparison to today where teams will need to travel between Miami and Seattle at times. All this to say it made way more sense to concentrate teams to the Northeast/Midwest., even if it meant using small towns.