For instance, if I am reading about Imperial Rome, I'd like to know what it looked like.
Of course, an obvious option is to go and watch a high-budget movie or TV show set in that period, and hope they've done their homework, but even if they have I imagine that many things would have been updated for a modern audience (hairstyles, hygiene?) or exaggerated for effect.
So it makes me wonder if historians ever publish benchmark visual reconstructions – by compiling all the written research on a period and translating it into images.
If so is there an official term for such things?
They certainly do.
The obvious one that comes to mind is Rome Reborn, an international long-running project that is recreating the city of Rome in 320 A.D. (judged by the project to be the peak of Rome's urban development) in 3d animation. They've been at it since 1997 and the project by now is quite advanced.
Of course, there are limitations. You won't see the kind of graphical fidelity high-budget films or video games have. It's rather large-scale and abstracted, so you won't see all the little details you might want. Nor can you expect to: because it's purpose is to catalogue what we know, rather than to present a speculative image, they leave out everything we have no evidence of. (Where those high-budget fictional productions would simply make stuff up to fill in the gaps.)
Still, it should give you pretty much exactly what you are looking for.
Here is a general overview video of the current state of the virtual city. That page also includes plenty of links to models of citizens, clothing styles, etc. Here's another one with narration by the project lead.
Have fun!