Very little light polution, yes. And, barring atmospheric events like volcanic eruptions (which did influence the weather globally, on a few occasions), probably a lot less polution to form clouds and the amazing sunsets our modern poluted atmosphere is so good at producing.
I spent two weeks camping in the countryside to dig up an Anglo-Saxon feast hall, and the location was remote enough that the light polution was pretty low. The stars were incredible, and I spent a lot of time thinking about how much more important they had to have been to the inhabitants of the site I was digging; they were bright, and the light polution was low enough to see some colors. And, most excitingly of all, I saw a shooting star almost every single night (one night, which was not a scheduled meteor shower, I counted 10).
No wonder people in basically every premodern culture believed the stars were magical. We would too, if we didn't have electric lights.
It would look a lot like it does when you go somewhere remote and far away from cities, or at a very high altitude like a mountaintop. There might have been some degree of light pollution from torches in cities, but it would have been nowhere near the level that we currently have. People in preindustrial times would have had regular view of the milky way in the night sky, it just was a long time before we as a species understood what it was.
hi! you may be interested in this post