You're going to want to read Goldsworthy's 'Complete Roman Army'.

Suffice to say, yes, the army did have ranks. Much as in a modern army, we can delineate these between the general staff who operated with the commander of the legion, and the field officers who were in command of the individual units of the legion.

First, let's consider the make-up of the legion. I'm going to go with the post-Marin legion here, and more specifically an early Imperial legion, for ease, as I assume that is what you are referring to. That is, a legion consisting of six cohorts, subdivided into 10 centuries each of 80 men, subdivided into 10 conteburnium(tent group) of 8 men each. So, roughly we can make the comparison of a legion=division, a cohort=batallion, a century=company, and a tent group=squad. What's lacking here is the modern platoon, which is commanded by an officer.

Each legion was lead by a legate, equivalent to a brigadier or a major general in today's armies. He might have a higher legate above him who served as the overall commander of the multiple legions. But that can complicate things, so let's move on. Below this we have the military tribunes, who were essentially parts of the general staff. Chief of these would be the tribunus laticlavius, the nominal second in command of the legion, although primarily acting as the chief assistant officer to the legate in command. This was primarily a political position held by a wealthy equistrian or patrician. Below this was the other five military tribunes who made up the rest of the general staff. While none of these tribunes were given default command of any cohorts, it should not be taken as a coincidence that there is were six military tribunes on the general staff and six cohorts in a legion. Because the legions were designed with the ability of the cohorts to operate independently, military tribunes were often put in command of a cohort that was dispatched independently. Thus, one could argue that these men could be equivalent to majors or colonels, although they weren't necessarily given responsibility for any actual troops in battle. Finally, the last was the camp prefect, who would be equivalent to modern Sergant Majors. These were the highest ranks that a man who was enlisted as a poor schlub could theoretically achieve, and was essentially in charge of keeping the men in line.

Now, let's move on to the actual commanders of each individual unit. There was no official commander of a cohort (see above), but nominally this role might fall to the most senior of the centurions, who each commanded a century. This was the basic rank in the legion, and could mean many things, some of which isn't completely understood. The first cohort's first century was commanded by the primus pilus (first spear), who nominally was the most senior of the centurions. The rest of the centurions of the first cohort were the Primi ordines, and were also higher rank than the other centurions. This order of importance likely was repeated in the other cohorts (as in, the centurion of the first century in each cohort was likely the most senior officer in that cohort), and nominally they would take command of the cohort in battle. In practice, this meant they were responsible for relaying orders from the legate, not necessarily more independent action, and we can assume that any cohorts dispatched independently would be commanded by a tribune. Centurions themselves walk a fine line between NCO and commissioned officers, and part of this stems from some confusion as to how one became such an officer. Goldsworthy argues that a large part of the centurions were basically purchased positions, through money or favours. These weren't necessarily rich people, but rather, young men whose family had just enough money and influence to get them a relatively prestigious (and well paying) job. However, there are certainly men who achieved these positions by enlisting as ordinary legionnaires. I think the safest bet is that their ranks were made up of a mix of relatively green but somewhat higher status men and older veterans who achieved their rank through loyalty and success in battle. Each centurion also had an Optio, probably equivalent to a staff sergant. This was the second in command to the centurion and nominally took over for him if he fell in battle. Below this were the Tessarius, basically third in command, and another sergant equivalent. Finally, we have the decanus, who was in charge of the tent group. This position doesn't appear to have been given more pay, and because the tent group was more a logistical position than a battle position, we can probably assume it was a prestige position designed to organize better discipline while in camp. Likely held by the most senior legionnaire in a tent group.

Finally, we have the auxilia. These were the non-citizen men, and were organized into similar structures of the legions. They would likely have similar officer ranks as well, although they were all organized as cohorts rather than full legions, and were likely armed and armoured either as light infantry or, perhaps more often, full clones of their legionnaire brethren. Cohorts would be commanded by Roman officers, called a prefect, who would be under the direct command of the legate. This was most equivalent to a colonel. The sub-officers and NCOs of these cohorts seem to likely have been drawn from the non-citizens rather than the citizens, but would all earn substantially less than their Legion equivalents.

There were other positions as well that were outside specific military duties. These were basically prestige positions such as horn blowers and standard bearers. Each cohort would have engineers, and the legion itself would have a medical staff.

While I would think it safe to assume you mean the Western Roman Empire, but this old answer covers ranks in the Eastern Roman Empire.

The nature of the chain of command in the Roman military is mentioned in Luke 7. The passage is mainly talking about something else, but what it says about the way military authority works makes it clear that the general principles were much the same in the Roman army as we think of them today.

The centurion was himself a "man under authority" (there were higher ranks); and, in turn, his expectation that his own orders would be carried out by the men under him (there would have been at least two ranks below him) was strong enough that he didn't need to inspect the work in person; he could just issue an order and immediately considered it done, and the men under him would take charge of the actual execution.