Well, they certainly like to be displayed as such! This ist the funerary monument of Caius Spectatius Priscianus and his family, late 2nd century, from the necropolis of ancient Celeia, modern Celje in Slovenia, at Šempeter. C. Spectatius Priscianus was a duumvir iure dicundo, one of the 'two men for speaking law', the highest municipal magistrate (like a mayor) of his town, and he is displayed here seated together with his wife, Caia Septimia Iusta, and his son, C, Spectatius Finitus, who financed the monument and was himself a duumvir. The family was part of the absolute elite of their area - if the absolutely extravagant tomb didn't already tell you that. He's wearing a tunica and a toga, as is his son, the whole family is wearing their absolute best.

There are several other examples, mainly from funerary art, that display high-ranking members of the local elite as a togatus, a toga-wearer, from all over the empire. This is an early example from Carnuntum, in Austria, of a soldier of the 15th legion, here displayed as a citizen in a toga. Soldiers like this one brought the trend of funerary commemoration in the form of elaborate tombs, stelae, mausolea and so on to all the corners of the empire, where this custom was emulated by the elites (in the hellenistic east and northern Africa there were significant earlier traditions of such forms of commemoration, but in the northwest, only rudimentarily and not as a mass phenomenon).

Here's the funerary stele of a decurio, a member of the local council of Celeia, Aurelius Gaianus, together with his nephew, Aurelius Maximus, both wearing a toga and holding a scroll - which is typical for this type of portrayal, probably signifying erudition and education, a symbol of their class.

Publius Anicius Baric lived in Chemtou, Tunisia, and was displayed on his funerary stele in a toga, and shown in the act of sacrificing - a symbol of his piety, a very important virtue for every Roman. This is a funerary monument for a couple from Speyer, in Germany (roman Noviomagus). I could continue this list for quite a while, because it was such an almost stereotypical way of displaying the dead. The togatus was a visual chiffre for saying: this dead man was a Roman citizen, an embodiment of all the virtues associated with that, such as piety, education, wealth, family. Moreover, since only citizens could wear the toga, and for a long time, not every inhabitant of the empire was a citizen, it was a clear mark of status. Romans were known as the gens togata, the toga-wearers. Wearing it was as Roman as apple pie is American.

Funerary display thus is a great way of finding out what people wore. In the provinces along the Danube, f.e., the stereotypical men are often paired with a woman wearing costume more associated with local traditions, local garb, probably to signify their connection to ancestral customs. But since this form of display is loaded up with that much symbolic ballast - it's the way people wanted either themselves or their ancestors remembered for all eternity, literally their identity set in stone - it's an idealized picture. It's not what they wore every day. Juvenal, the great satirist, gets to the point very quickly in typical fashion:

Pars magna Italiae est, si verum admittimus, in qua nemo togam sumit nisi mortuus.

'There's a big part of Italy, we readily admit, in which nobody wears the toga unless dead.' (3,71f.) Clearly either refering to literal burial in the toga, or being displayed as such only on your funerary monument.

And Martial, while describing a rural idyll, puts in the line:

Lis numquam, toga rara, mens quieta

'Never a lawsuit, seldom the toga, the mind at ease'. (10,47,5)

A toga is neither easy, nor cheap, nor comfortable to wear. The material is costly, so is keeping it in pristine condition, which also means fulling it from time to time, to keep it shiny. Magistrates and members of the council would probably wear it to official functions, for sacrifices, feasts or celebrations, but it was certainly not the every-day garb of choice for every citizen. It's like an expensive business suit, or ceremonial uniform, the sunday best you wear to church.

That being said, as an ideal for the male Roman citizen and one of the very symbols of romanitas, of romanness, it penetrated every corner of the Empire, being evident in all the local traditions. Its symbolic value was understood and could be used to showcase your status and your identity from Scotland to the Red Sea, from Morocco to the Crimea. From the third century on, however, the distinctive value of Roman citizenship was diluted when every free inhabitant became a citizen in 212, and more and more, clothes that stemmed from military traditions became the norm that was associated with a Roman, even though high officials continued to wear the toga until the end of the Western Empire, and beyond.