Historically we see the development of trouser and trouser-like things amongst people who have horse riding as a significant part of their lifestyle. Without thigh protection, riding a horse for a long period of time isn't particularly fun. We typically see women wearing trouser-like garments - or skirts that can be easily adjusted for horseback riding - where they are expected to ride. Even looking at people who wear extremely gendered clothing like upper-class Victorians, women's riding habits (even when typically riding side-saddle rather than astride) generally feature long trousers underneath.
So, going right back to "the beginning". We don't have a wealth of detail on ancient clothing, partly due to the problems of preservation, but we can see various groups of Iron Age Europeans wearing breeches or braccae. These are especially common amongst nomadic steppe cultures. For example, it has been suggested by archaeologists that younger Scythian women - who were more likely to be warriors - typically wore trousers, while older women tended to wear skirts or robes over leggings. Both these forms of clothing were convenient for horseriding, but the trousers more so for someone who was expected to be able to rapidly get on and off a horse.
There's some interesting classical Greek feeling that these trousers are a feminine piece of clothing. The people who wear them are mocked for being effeminate, and even Xenophon on horseriding tells his readers to rearrange their skirts under their butts in order to cushion themselves rather than adopting anything that might be seen as too foreign. There are claims that that they were an invention of women - whether that's Medea, or the Persian rulers Atossa, Semiramis or Rhodogyne, all three of whom are credited with inventing the fashion in order to disguise their sex and blur male/female differences (and all of whom are also female warriors). This is - unsurprisingly - particularly disturbing to the classical Greeks, whose cultures typically have extremely differentiated gender roles. Being unable to know if someone was male or female - i.e. if they were wearing trousers - was an alarming proposition.
So why does this change in Western Europe? How do they go from being a feminine/unisex garment to one worn specifically by men? The answer is the increasing importance of horses in warfare. Women typically aren't involved in combat (although where they are, it's not uncommon for them to adopt "masculine" clothing, whether that's to do with gender performance or practicality), but men are. Trousers - which make it practical to ride - become masculine through this association and gain social status. We can see this shift during the late Roman empire. Tacitus tells us that Aulus Caecina Alienus wears breeches when riding in 69 AD, something which is clearly inappropriate and Gaulish. However, by the 300s, the Historia Augusta describes Severus Alexander as wearing white trousers rather than the typical red, and references trouser use inside the army. This could be related to Roman cavalry, but also the needs of soldiers further north - short tunics simply don't cut it in a Northumbrian chill.