Okay, so as a reenactor Roman clothing is one of those things I love talking about, so this should be fun to answer.

A lot of our information tends to come from contemporary visual depictions of clothing, arms, and armor. And for the longest time, this was all there was, and that can be seen in the historiography in the study of Roman armor and clothing, most notably through the evolution of the reconstruction of the infamous banded plate armor known today as Lorica Segmentata. Originally, 16th-19th century interpretations were based almost purely on what we had: art, notably Trajan's column, until the Corbridge find which allowed Russel Robinson to reconstruct it, until reenactors made the thing and figured out it was impossible to wear, which led to the Peter Connelly reconstruction and eventually Bishop and Coulston's reconstruction. And this brings up the problems with studying Roman art, because what art depicts doesn't always coincide with archaeology, or actually work, for that matter.

There are basically two dominant schools of thought on Roman art and how it relates to reconstructing equipment and archaeology. The first and the academically respected one is the British school, which values art as a resource but considers it always subservient to the archaeological evidence. The other is the view that art is just as valid or more valid than archaeology, which presents some problems. I call this the Italian school because it tends to be Italian authors that follow this viewpoint, but I don't know what this school of thought is actually called. But nevertheless, it's useful to discuss the problems with the idea that art is just as valid of a source for this information as the archaeology:

1. Classicism. Roman artists were trained to create their works in a certain way and to allude to other works with their art, much like Roman authors were trained to write a certain way and allude to older Roman and Hellenic works in their writing.

2. Experience and source. In terms of portraying things like the Roman army accurately, many artists had either no desire to do so, or insufficient training or experience of the Roman army's appearance. Generally speaking, the provincial works tend to be more accurate than the major works in Rome or Constantinople, because they had exposure to more than just the imperial parades wearing nothing but dressed up tunics.

3. Stylistic choices. Many artists chose to simplify, or over-elaborate, appearance in their art. Trajan's column is a great example, where the Lorica Segmentata is stylized and couldn't possibly function if constructed that way, and lacks the intricate details of how it was constructed. Or how the Roman legionaries are always depicted wearing segmentata so the audience knows who the Romans are in the work.

4. Purpose. Again, art had different purposes. Some were private commissions with specific goals from their patrons. Others were imperial commissions designed to visually present the auctoritas of the emperor. And others existed too: one of the arguments about the "leather muscle cuirass" in art is that the pieces depicting flexible, cloth-like armor come primarily from Theatres, suggesting the artists were depicting what one might expect to see in the theatre as a form of advertisement.

5. Interpretation. It can be hard to determine what something in art is. For example, the "horns" on the helmets of the soldiers in the Arch of Constantine. Many purport they're just that: metallic fixtures in horn-like shapes. But that relies heavily on 1. the lack of detail in the relief and 2. the assumption that the unit depicted is the "Cornuti", which may not even mean "horned ones" at all (it could be "horn-blowers", "flankers", or totally unrelated stemming from a Germanic word.)

Generally speaking, clothing and how clothing was worn seems to be more accurately portrayed in art than military equipment. In, fact, it's not until the late Roman era that things really start to get fuzzy, due to a mix of lack of sources and increasing complexity. Court dress becomes a thing in late antiquity, where court functionaries start having to wear specific clothing associated with their station. A lot of that information is missing until the reign of Nikephoros I and Constantine VII, when we have documents like the De Ceremoniibus and the the Kletorolougion in the 800's AD. For more on Byzantine court dress, I strongly recommend Timothy Dawson's By the Emperor's Hand which goes into great detail on outlining the different roles and patterns of dress for both court functionaries and military men. For late Antiqiuty we by and large have to rely on fragments from multiple sources, namely the law codes of Theodosius II and Justinian, as well as Lydus De Administrando Imperii. Plus a few others. Imperial Rome is far better sourced, and by and large doesn't have the issue of certain patterns of clothing pertaining to certain roles and ranks, with some exceptions.

Art comes in in figuring out precisely how it was worn. For example cloaks, which often have complex patterns of folds because they were folded and pinned a certain way. Archaeology, sometimes, provides us this context too: for example tunics with long stripes (clavii) running down the front, in the late Roman period, are typically characteristic of Children's and Women's dress, not Men's dress. Archaeology also conflicts with art too: we've never found a tunic with a neck or arm slit large enough to wear it only on one shoulder, like we often see in art.

For further reading there's a mix of books that I'd recommend, but in terms of Men's dress the best is Graham Sumner's Roman Military Dress, as well as the compilation work Wearing the Cloak: Dressing the Soldier in Roman Times. Despite their military focus, they actually cover a lot more than just military dress.

This is a really big field so I tried to give a simple answer, but feel free to ask more questions if you have them because I can probably answer them or find you an answer to them covers them.