He wasn't: Constantine did not have a beard, for instance. I do admit there was a long stretch after Trajan who sported beards --Hadrian, Antonius Pius, and Marcus Aurelius being enough to start a trend-- but if you take a look at Second, Third, and Fourth Century coins, clean-shaven is definitely still represented.

Edit: do not don't.