There is no singular answer to this question because different cities were founded on very different legal systems. If a city was part of a lord's personal domain then he had a great deal of control over who was in charge and why. But most cities weren't part of a nobleman's domain. Many cities were much older than the middle ages and already had an established power base that later kings and emperors negotiated with to include. As a result, many of the old Roman Civitas were governed by their own charters, which controlled such things, and were bound to a king directly by such charters.

Even today some uncommon things persist as a result. For example, while London is a bustling metropolis and capital of England, the City of London is a very small bit of the larger metropolitan area that still has a separate government governed by a charter significantly older than the Norman Conquest. It seems that the first mayor didn't pop up until 1189, but no one knows if he was elected, appointed, or just declared himself to be of equal authority to the Sheriff who enforced the King's authority in the city. By 1215 the city had settled on a system that elects a mayor on a system where the guilds and major corporations get a say roughly equal to that of the residents.

Many of these old towns had the civic powers that be fail to survive the transition from the Roman world to that of the Middle Ages. In these cases it wasn't uncommon for a Bishop to end up in charge of the day to day functioning of the city. In these cases they tended to wear multiple hats. The best example of this is the fact that the Pope is also King. Even today he is the King of that little theocratic state in Rome, which is technically a separate office from the Pope, but when he became one he became both so he governs as King just as the Prince-Bishops of the Holy Roman Empire were both rulers of city-states and Bishops of the church. Bishops were traditionally either appointed by the King or by the Pope depending upon the rules of the day. Which rules were in place at any given time and in any given place was a matter of much contention between the crowned heads of Europe and the clergy.

I guess that the answer to your question is all of the above. The ruler of a city might be a hereditary lord if the city isn't "free", many chartered cities had rules for elections (but not for popular vote to determine who ruled), and if the city was "free" but didn't have a charter it was ruled by a sheriff appointed by the King or a bishop at least nominally appointed by the Pope.