Depends on the sport, the earliest recorded rules of numbers being used was for football (soccer) in Australia but there are pictures of numbers being used in sports prior, just not officially a necessity or on the rule books. Informally football (soccer) would have numbers 1-11 for the players and then higher numbers for substitutes. Football (soccer) and American Football did not introduce widespread rules requiring numbers until the 1950s. Baseball was one of the earlier sports, appearing in 1907 in independent leagues and then the first Major League team adopting them in 1916 but was abandoned until the St Louis Cardinals wore them on their sleeves. The Yankees started wearing them in 1929 and it became standard practice once they started doing it. I believe the last team was Philadelphia who at first only wore them on away games because they were forced to. Basically by the 1950s every major sport had them, but they would tamper with where the numbers were (front, back, sleeve, etc) and which numbers were allowed (basketball had very strict rules on numbers allowed).

Mostly it came out of necessity of identification, radio, and keeping stats. It wasn't uncommon for baseball players of the late 1800s and early 1900s to try to sneak back into a game they had been kicked out of without an umpire noticing, for example. And record-keeping is particularly hard for the statistician if they were expected to know and remember what each player looked like and try to keep track of substitutions, particularly in a sport like American Football where players swap out all the time without people noticing even when they do have numbers.

Not to mention retiring a number is kind of a cool tribute!!

In ice hockey, the Pacific Coast Hockey Association was the first to use numbers for players when it was founded in 1911. Frank Patrick, the president of the PCHA and co-founder with his brother Lester (both also played), was a major innovator in ice hockey through his league. The basis for adding the numbers came after he saw a track meet and noticed that the runners all had numbers to help identify them easier. He felt this would be useful in a sport like hockey, where players moved at a considerable pace and some had trouble keeping track of who was whom. I don't know off hand, but I believe the National Hockey League (it was based in Central/Eastern Canada, while the PCHA was in Western Canada) took a few more years to adopt the same format.