In the time of levied armies, "specialized" (such as they were) mountain warfare units would have been raised from the mountainous regions of your lands or, if unavailable, you would've hired mercenaries from such lands.

The earliest example I can think of, though hopefully someone is aware of more, would be the Agrianes who accompanied Alexander the Great. They were a Paeonian-Thracian tribe from what is now the province of Pernik in Bulgaria. They were fierce warriors who were accustomed to mountain warfare due to their native landscape. Both Arian and Robin Lane Fox heap praise upon their fighting skill and they were present for many of Alexander's decisive maneuvers. More specifically to mountain warfare, they are recorded as being particularly adept at climbing and scaling walls, skills that definitely came in handy when Alexander was campaigning through Afghanistan.

I'm not sure if that's what you're looking for, but the Agrianes were overall excellent fighters who also had skills suited for mountain warfare.

Most of the mountian warfare units in Western Armies were created after the first world war, especially in light the intense high-altitude battles that took place between the Italians and the Austrians.