You’re right that hats are rare in Roman art. There are statues of the god Mercury wearing a soft cloth traveller’s hat, which makes sense since he protected travellers. I suspect a lot of everyday travellers would have worn similar cloth or straw hats to keep off the rain or sun, but these were more worn by commoners, and not considered the kind of clothing you wanted to show off in a portrait.

The people having their portraits made are the elite and are usually pictured in formal wear (toga) or as some hero or divinity (usually nude). Hats weren’t part of formal attire, but Romans would pull the folds of their togas over their heads when mourning or sacrificing to the gods. You can see Augustus on the Ara Pacis for an idea of what it looked like.

Women, by the way, also used veils or the folds of their stolas to cover their heads, often held in place with tiaras or other headpieces. Some rich Romans rode in litters too, partly for protection against the sun and rain.