The problem isn't that the single act caused the defeat of Napoleon but rather what it prevents.

In Napoleonic warfare, heavy cavalry existed for one reason, to be a battering ram against infantry in order to create a hole that would be taken advantage of by your infantry and light cavalry. And so, with Neys charge, there are several problems.

First is that of reinforcements. The charge Ney made was very classic Ney, impetuous and foolhardy. With a sudden and pointless attack, there is little communication (as it wasn't Napoleons order to charge) to the other corps to follow up. As such, there is no chance of exploitation of the British center.

Second, it had no impact. The charge against the British artillery was wasted as the British crews were able to run into infantry squares and wait the cavalry out. Worse the French cavalry didn't even spike the guns which would have made the British artillery useless.

Third and worse, the bolt was shot. Heavy cavalry is made of heavy men on heavy horses with (sometimes) cuirasses and other arms. They wear out on,the charge because they are heavy, big, and unwieldy. As a result, the French cavalry, even if they survived as a whole, would be exhausted. The bolt is shot, they are tired, the horses are exhausted. The heavy cavalry charge is a final act, a coup de grace to a defeated enemy. And Ney did it in the middle of the battle in the strongest of the British positions to little effect.

Ney for lack of a better word, destroyed Napoleons chance for victory due to the poor use of an amazing and powerful unit and tactic.