It is important to understand that different cultures may have utilized the domesticated horse differently. Though the earliest mention of horses comes from cave art dating to 30,000 BC, evidence (see here) suggests that the earliest domestication of horses occurred in Kazakhstan as far back as 4,000 BC and as recent as 2,000 BC (there is a deal of discrepancy). In any event, we are fairly sure that the domestication of horses occurred in the Eurasian valley. Archaeological evidence from Egypt, Assyria, and India suggests that many eastern cultures had access to the horse.

Some of the earliest evidence of the use of the horse comes from the Botai culture (around Kazakhstan). They utilized the horse to hunt herds of wild horses. Archeological evidence for this includes their waste pits (most bones coming from the wild horse) and pottery (some of which is saturated with the milk of horses). This ultimately led to more permanent settlements.

Later on (around 3,500 to 3,000 BC), horses began to appear farther from the Eurasian steppes. Evidence suggests that the horse reached central Europe.

Horses were often seen as symbols of power, even before their domestication (evidenced by tools shaped into horse-heads and such). After domestication, horses were not only utilized for agriculture and hunting, but also for warfare. A Chinese military code from 2,637 BC mentions the cavalry as part of the army. Horses may have been used both to pull chariots and to mount. A Sumerian illustration dating from 2,500 BC depicts horses pulling some sort of wagon.

Cultures who were able to make use of the horse probably had a great advantage over those who did not. For example, the collapse of some “Old European” mining towns in the Balkans has been attributed partly to raids by mounted Indo-Europeans. However, this may be just one factor contributing to the collapse.

For more detailed information, I suggest checking out this podcast about the domestication of horses: http://www.cbc.ca/quirks/episode/2009/03/07/horsey-aeology-binary-black-holes-tracking-red-tides-fish-re-evolution-walk-like-a-man-fact-or-ficti/

Also, you may want to check out this cavalry website: http://www.cavhooah.com/info/sections/cavalry-history/

Horses were certainly domesticated very early on, but there were some obstacles to making a true cavalry force. The first domesticated Asiatic horses were very small, and could not easily support the weight of a grown man with armor. Because of this, the first principle method of horse transportation was almost certainly by chariot, a natural evolution of the wagon. The first chariots we have are from the Sintashta Petrovka culture in western Russia around 2000 BC. We also have horse bits and riding equipment that suggests that horses might also have been used for cavalry around this time. That is however somewhat theoretical. The first concrete evidence for horse-riding in my mind is in Mycenaean Greece, where pottery images of horsemen are common-place: http://www.salimbeti.com/micenei/chariots.htm

Although I would not count them as a true mounted cavalry force as horse-riding was restricted to scouting roles and recreation(this is similar to the situation in many of the other great Bronze Age empires, who did not even view horses as a noble animal for riding, but rather a dangerous and uncomfortable alternative to donkeys.).

There is some theorizing, such as TurinTurambar704 says, that the Indo-European steppe nomads used horses to bring down the 'Old European' cultures. This view is mostly advocated for by Marija Gimbutas with her Kurgan hypothesis. The Kurgan hypothesis is a relatively sound explanation of the spread of Indo-European languages, but the whole clash between the farmers of 'Old Europe' and the horse-riding steppe warriors comes more from her later feminist views than actual history, so I would take that idea with a grain of salt.

This is not to say that there was not horse-riding in the steppe, just to say it was not the primary method of travel compared with foot travel, wagons, and sleds: http://stepnoy-sledopyt.narod.ru/history/esp/6.jpg (in the top row are some images of horses being used to pull skis, or sleds)

Also(and this is my final main point)The early steppe was not like it is today, it was much lusher and more amicable to agriculture so the cultures who 'invented' horse-riding didn't use it a lot because, there lifestyle did not wholly depend on it like the later Scythians or Sarmatians.

I would say all things considered the first true cavalry force in a military sense would be the warriors of the Andronovo culture around 1500-1300 BC, and the first recorded military cavalry force the cavalry arm of the Cimmerian armies of the 10th century BC.

I hope I answered your question sufficiently :)

Also, TurinTurambar704, where did you get the date of 2,637 BC? I do not think there was any writing at all in China at that time, so how can there be a military code?

Just curious, that's all.

For sources this comes from a lot of personal research, mostly from academic documents and journals from Academia. I can provide sources if you want any clarification.