I am so glad to come upon this question! I am in the final few months of completing a Master's degree and my thesis is a study of the highest ranked generals of the fourth century Roman army. I just finished writing a section about this exact topic.

Only a single Roman 'magister militum' (the Latin title for the highest ranked generals of this period) is said to have some through from the lowest ranks of the army. This man, Arbitio, is said by Ammianus Marcellinus, a contemporary historian, to be so ambitious he was not content with rising up from the lowest ranks of the army to such a high position (at the time he was the most favoured magister militum of the emperor, Constsntius). However, I don't know how much I believe this. Uniqueness is always tricky when it comes to history. Things generally follow trends, and if someone is attested as the only person to have done something, it makes me suspicious. What also makes me doubt the veracity of Ammianus in this claim is that he had a powerful dislike of Arbitio. Ammianus himself served in the army, and the magister militum he served under was a man named Ursicinus. In 359/early 360, Ursicinus somewhat wrongly received the blame for the loss of the Roman city Amida during a invasion by the Persians, and Ammianus believed that it was a group spearheaded by Arbitio that conspired to convince Constantius to blame Ursicinus. So Ammianus was quite hostile to Arbitio and may have lied that Arbitio began at the lowest ranks of the army because that would maybe imply he was low class and poor. This might have been intended to tarnish Arbitio's reputation.

On the other hand, it surprises me that more soldiers could not rise through the ranks to high positions in the army, but if it happened it definitely has not been recorded in the ancient sources.

So where did the generals come from? We actually have very little to go on. Most, about 21% of the 52 magistri militum of the fourth century, are recorded to have occupied middle tier ranks in the army. These included the positions of 'tribunus', 'dux', and 'praepositus'. These offices generally commanded a single unit of soldiers, maybe around 1,000 for the tribuni and praepositi, and maybe some multiple 1,000 person units for the duces, but they were border guard units and of lower class. The tribuni could also be given special tasks like reconnaissance and intelligence gathering, and supporting the higher ranked officers in their duties. These positions were already fairly high in the military, and the emperors Valentinian and Valens were actually raised directly from these positions.

However most future magistri militum we're promoted into the 'comites' rank, either as 'comes domesticorum' or 'comes Rei militaris'. The comes domesticorum was the leader of the 'protectores domestici', a kind of officer training school. The comes domesticorum could also lead soldiers and conduct important missions for the emperors. The comes rei militaris would be in charge of smaller armies made up of multiple 1,000 man units. They would be responsible to the magistri militum but not necessarily under their direct oversight.

The question that still bugs me is why we don't know more about where these men came from before the already pretty high rank of tribunus or similar. Some we're definitely recruited from foreign barbarian armies, and if they already had proven experience that would justify them being fast tracked into the office of tribunus, but this can't apply for them all. Maybe the ancient historians were not aware of these men until they already distinguished themselves in these high ranks. Or maybe it was considered uncouth to record their early deeds in the rank and file. it doesn't seem possible that they were 'old money' aristocracy that got special recruitment, because almost none of them had big wealthy families that we know of, and there is only one case in the fourth century of a magister-ship being passed on somewhat hereditarily, and that case was really based on merit and need rather than favoritism. I don't know if anyone will figure it out completely satisfactorily.

Hope that helped! Feel free to ask me anything about the fourth century generals and if it's known to us, I can answer it! Also sorry for any spelling mistakes, I wrote this while cramped on a bus