Source is The Horse, the Wheel, and Language, about archaeology and the origin of Indo-European language.

In the book, he posits that the Proto-Indo-Europeans were the first to domesticate the horse, and so of course the first to use it in battle.

To put this in historical context, the Proto-Indo-Europeans lived in what is now Ukraine and southern Russia, around 3500-4000BC. They didn't live in cities themselves, and they probably didn't have wars in the same way that states would later on. Most battle would be small-level raids by the men of one village/small social in-group against another village/small social in-group, and there wouldn't have been big battles with thousands of men on each side.

It's theorized that horses are one reason that Indo-Europeans, and their language(s), were able to spread so far across Eurasia. However, most of that expansion probably wasn't in the form of bloodthirsty hordes from the steppe pouring into Europe and the Middle East and India, slaughtering the peaceful matriarchal locals from horseback - which became something of a meme among the less-informed after Marija Gimbutas, one of the important early proponents of the steppe origin of Proto-Indo-European, expanded upon the theory by claiming that the strongly patriarchal Indo-Europeans destroyed native European traditions of agrarian matriarchy (there's possibly small grain of truth there, but it's been taken far beyond the evidence supports). I'm sure there were armed confrontations between Indo-European newcomers, and locals who weren't happy with the new neighbors, but I don't know if you could firmly say that the Indo-Europeans were the first to use horses in what we would later categorize as proper battles, rather than skirmishes.

If not the Indo-Europeans, then it was probably one of the settled peoples in the Fertile Crescent and surrounds who first used horses widely in warfare. Interesting to note is that chariot warfare preceded the widespread use of individual warriors mounted on individual horses, probably because before later technological developments in individual horse-riding equipment, it was a lot easier to fight from a chariot's platform than it was from horseback.

Addendum:

No "nation" was the first. The Proto-Indo-Europeans gave their language, and aspects of their culture, to most of Europe, the Eurasian steppe, Iran, and India, but no single group of people in the modern day can say "hey, WE are the true heirs of the Proto-Indo-Europeans!", even if some of them may try. Perhaps the Scythians and Sarmatians would have a strong claim if they still existed*, being the last Indo-European people to continue unbroken the tradition of steppe nomadism (and in the same location as the Proto-Indo-Europeans), but even by the earliest historical mention of the Scythians, every aspect of their culture had changed drastically from PIE times.

*I know about Ossetians but it's probably better to consider them as a Caucasian people speaking an Iranian language... which is a good analogy to what happened with the rest of the Indo-European peoples who left the steppe.