While there obviously would be variation between different countries, I wrote a previous answer specifically about the Byzantine (Eastern Roman) military, which might be of interest for you.

I can speak for late-medieval Italian city-states in the 12th century (called Comuni, sometimes rendered in english as "Communes"), which is my area of greatest expertise. I hope someone can describe the situation in the Royal armies of the kings of France and England, I imagine it is somewhat analogous but with some key differences. I can also add that no, it wasn't akin to ours at all, and the only way to rise through the ranks was if you came from a background of privilege.

In Communal Italy, cities would elect a Capitano del Popolo, or "Captain of the People", who was in charge of the city defense (and, when necessary, conquering other cities). Milan, for example, had Seven families, each charged with the defense of one of the city gates, and from these the Captain would be elected. The families were chosen because they were wealthy enough to fund the defense, and indeed many held vast estates, castles, and even the lordship of some towns, outside of Milan.

(If anyone was wondering, they were the Houses of Della Torre, who guarded Porta Nuova, Argentea, who guarded Porta Orientale, Da Baggio, charged with guarding Porta Comasina, Da Busto, guardians was Porta Ticinese, Grassi, of Porta Vercellina, and Caronia, of Porta Romana. Source: La Vassalità Maggiore nel Regno Italico Andrea Castagnetti, Roma, Viella, 2001)

Should war break out, the families would call their land-holding Vassals to arms. These vassals would fight as knights, either on horseback or on foot (depending on the situation) in companies under the command of their lord (Most of the time. A particularly powerful lord, like Della Torre, or the Lord of Massino, better known by his title; Visconti, would have several companies of knights, probably each commanded by a family member).

In the Lord's household, as well as, to a lesser extent, in the household of some knights, would be servants called Sergeants. Their job was to protect their master in peacetime, while in times of war, some could perform light cavalry actions, however most would dismount before battle and would stand at either end of a line of ten conscripts or so. Their job would be to push against the ends of the line of conscripts so they they couldn't move, making sure that they didn't break and run, and also making sure they they presented a solid mass of spears to the enemy.

Conscripts were mostly made up of the inhabitants of the city, and able-bodied serfs from the country surrounding it. Their job was to not die. There were also free citizens of the city who mostly fought with crossbows that people will complain that I haven't mentioned. We could open a whole new chapter on them, but I'm grouping them here.

For your average conscript, the only way to make it to a higher rank would be if a Lord took you into his personal employ because you were big and sufficiently thuggish (as described in La Catedral del Mar, by Idelfonso Falcones, a very well-researched historical novel). A Sergeant, on the other hand, might be able to steal enough plunder to buy his freedom and begin a different career, however, there is nothing to indicate that they would want to (life was relatively comfortable when your job was to be the top dog's thug). Knights could also amass personal wealth through plunder, but those that really stood to gain in rank were the Lords, especially minor ones (as major ones often had their interests tied with a major city, and should that city fall...). Through warfare, they could acquire new estates (and consequentially, revenue streams) as well as develop undue influence, especially when the middle ages started giving way to the renaissance int the 14th century, and members of the minor nobility could find ready employ as mercenaries.

If you'd like to read more about communal warfare, I'd suggest The Lombard League by Gianluca Raccagni (Oxford University Press, 2010)

The typical story of a member of the minor nobility who rose in the ranks is that Giacomo Attendolo. Son of a minor landholder in the Romagna region of Italy, according to Paolo Giovio, his (admittedly biased) biographer, Attendolo was ploughing his father's fields when a company of mercenary knights rode through. Giacomo stole a horse from his father's stables and rode off with them. Amongst the mercenaries, Attendolo soon gained a nickname; Sforza, meaning "Strength". The rest is history. He rose through the ranks of the company, eventually becoming one of the most successful generals in Italy. His son would follow in his footsteps, and bead the armies of the Duke of Milan. He ended up marrying the duke's daughter, and the throne passed to him when the old Duke died childless.

edit: typos