You're right! Well, sort of. It's essentially clerical garb, since students at the first universities were technically all clerics. In the US, academic dress is fairly standardized across all universities and only worn for commencement, but this is not the case in Europe, where there is significantly more variation and it can be required during other times of the year. This is also where the various other features come from (master's hood, doctoral gown) which demonstrate academic rank.