Oh boy, I was chomping at the bit when I saw this question....here we go!
The evidence is quite clear that bitting has been around for a very long time but it is not the only method out there for controlling horses. Nose rings and bitless bridles (also called hackamores) have also been used in the past and hackamore usage continues to the present day. This begs the question why was bitting the most popular method of control throughout history?
To start off, we should look at how a bit functions as a control device. Horses have a number of more sensitive spots on their bodies, more so than usual as they are noted to be able to feel flies landing on their backs. Foremost among these are the horse's mouth, ears, eyes, and nose. A bit fits into the gap between the horse's front and back teeth, and depending on design, applies varying amounts of pressure through contact points on the horse's mouth/face. Snaffle bits, which feature reins connected at the ends of the bit, apply equal pressure on the mouth as on the reins, which is particularly useful for riders who are still learning the appropriate contact to have with the mouth. Curb bits feature the reins connected at the bottom of a shank. This transforms the bit action to something like a lever. Contact on the reins is multiplied by the lever action which allows a lighter touch on the reins to have a greater impact. Our third type of bit (or rather bit-less), Hackamores, put the pressure distributed over the horses nose and under the chin. There have been variations of these designs throughout history, but most traditional control devices can be broken down to these 3.
The problem with hackamores, from a mechanical perspective, is that the pressure is too distributed on the horse's head. Horses with no training, often called green or unbroken horses, and unused to any contact on their face will respond to the hackamore quite well. Horses with considerable training and eagerly responsive to aids are also likely to respond to hackamores. Horses in between, or those with an attitude, may be apt to ignore the rider's cues which is why a bit is far more effective at controlling horses due to the ability to focus pressure in a sensitive area. In a few cases, the hackamore is perfectly fine and sometimes better, but bits are more effective overall (when properly utilized).
Now onto the historical archaeological record (we're talking about things that predate writing here!).
Horses are believed to have been domesticated anywhere from 5000 BCE to 3000 BCE. They were NOT domesticated for the purpose of riding (see here for my previous answer on the subject). Horses were a food source like cattle or sheep or goats, but eventually some brave soul decided to try and ride them. The evidence for their domestication is in the bones and garbage heaps left behind by human cultures, particularly on the Eurasian steppes. Of these sites, Botai is one of the most well known (but still debated as the origin of domestication).
At Botai, horses make up over 99% of the animal bones found at the site. At other nearby Botai-culture sites, horses have been found to make up 90% of recovered bones. It is clear that the people of the site relied on horses to a significant extent. The site itself is dated to be from around 3500 BCE. Among the bones recovered, David Anthony analyzed teeth for bit wear. His work The Horse, The Wheel, and Language goes into depth on the methodology used to identify bit wear which I won't go into, but suffice to say he experimented with organic bits as well as metal ones. His experiment demonstrated that organic bits, ie those made of leather, bone, hemp, or horsehair rope, do cause bit wear on the teeth. The softer materials were noted to cause less bit wear than bone bits, but the wear was still evident after 150 hours of riding.
While bits have not been discovered at Botai itself (or at least nothing that has been identified as a bit), there was bit wear suspected on the various teeth he had analyzed. It is probable that if Botai used bitting, then they would have likely been made of one of the organic materials which may not have survived to the present day. However, Anthony's methodology has been criticized for seemingly not taking into account the possible other factors that can contribute to wear on teeth, including the site formation itself or abnormal biological conditions that could occur in a horse's lifespan. Additionally, age variation in the skeletal remains at Botai do not seem to indicate husbanded rather than hunted populations.
As for the introduction of metal bits, the earliest known specimen dates to around 1700 BCE. An equid burial site in Israel (Tel-Haror) was found to have a metal bit that was likely used on a donkey rather than a horse. It is logical that metal bits were simply the next step in bitting technology as organic materials have been documented among other cultures in more recent times. In conclusion: when did we first use bits? Hard to say, because organic materials tend to not fare as well in the archaeological record.
