The first people to use massed units of 'true cavalry' (i.e. men fighting on horseback) were the Assyrians, in the time of the Neo-Assyrian Empire (911-612 BC). The concept spread from there.

Generally, cavalry combat is not as spectacular as you're describing. A cavalry charge mostly relies on psychological rather than physical impact; if the target body of infantry remains steadfast when the charge hits, that charge has already failed. The intention is for the onrush of horsemen to make the infantry waver, so that the actual fighting will be between mobile mounted warriors and scattered, panicked, fleeing men on foot.

Indeed, charging into a tightly packed mass of infantry with cavalry is famously impossible. The commonly cited example is the French heavy cavalry's failure to break the English infantry squares at Waterloo. The problem is that a horse will not run where it does not believe there is room for it to go; when faced with men standing shoulder to shoulder, horses will not bowl them over, but will balk and stop in their tracks.

For this reason, a lot of early cavalry (including the mounted forces of the ancient Persians and Greeks) was hybrid 'medium' cavalry, heavily armoured but armed only with bows/javelins and swords. They would charge an enemy force to see what it would do. If the enemy stood its ground, the horsemen would throw their javelins and shoot their arrows and wheel about, but if enemy wavered or broke, they would charge home and hack them down. It's not until the 4th century BC that we see cavalry armed with lances, whose only purpose was to charge into the fray.

There's a lot of specialist literature on early cavalry, but for a solid introduction I would recommend Philip Sidnell's Warhorse.