This question has bugged me for years. I've always assumed that they would band together in groups based on who's lord was who, but I do wonder if all the ones who, say, carried axes got into one group and all the ones who carried swords and shields got into another. I know that archers were separate, but I don't know about spearmen or swordsmen, and if they were kept separate from one another, or if it was more disorganized than that?
Often Men at Arms refers to men who fought as knights but had no title ('gentlemen'). From context, I assume you are referring to non-knightly infantry Is that correct?
medieval armies could and did organize themselves differently. When and where are you thinking of?
My error, sorry. I was referring to the foot soldiers, the infantry levies, not the mounted men at arms.