Romans lacked the popular mass media that we do today, so it was very difficult to "advertise" and respond to changes in fashion. If some dude in Ravenna thought it was cool to wear his toga with a feather pinned to it, letting people in Carthage - or even in Rome - know that feathers were cool, was near impossible. That's one aspect.

The other aspect is that in a multilingual, multiethnic, and very stratified society, the outfit one wore had the practical purpose of reflecting one's position in society. If you were an emperor, for example, you wore a toga entirely purple. You may remember how in the 90s and early 2000s, it was "cool" to dress "street". Well, in Roman society, dressing "street" would mean that people would interpret you as literally being of the street. If you dressed in the clothing of general (gold lined toga if I remember correctly), people would assume you were a general. Changes in fashion every 50 years would've interrupted the purpose that clothing had as a visual indicator of societal status. You have to remember that some Roman outposts might only see an official from Rome every 5 or 10 years or so, and if he rocked up with frosted tips and a leather jumpsuit instead of the traditional dress of a bureaucrat, people might wonder what on earth is going on. Of course, as the centuries went on, fashion did change, but very slowly. It became acceptable to wear "military" clothing (we see this in early Byzantine fashion), and at the end of the Empire, it was even okay to wear trousers in some circles.