In order to fully answer your question, I think it's worth providing a (very brief) history of mapmaking leading up to the Early Modern period.

Maps (or graphical representations resembling our modern-day maps) have existed for millennia. Perhaps the earliest philosophical work we have on cartography is Ptolemy's Geography, written in the second century. Ptolemy conceived of cartography as a scientific endeavor, where space was measured in terms of latitude and longitude; however, the technological advancements Ptolemy would have needed to execute his ideas in full would not be invented for centuries.

Ptolemy was largely forgotten by mapmakers during the Middle Ages, whose work was more conceptual in nature. Key to this time period is the conception of space, not in terms of borders, but in terms of centers of power. A lord’s power was centered in his castle, and it radiated out to the surrounding village. As one got farther and farther from this locus, the degree of the lord’s influence diminished, until ultimately, miles out, the land became a sort of frontier, with no discernible, distinct border marking territory ownership.

In the Early Modern period, the "rediscovery" of Ptolemy's Geography and the advent of the printing press meant that maps were more numerous, more popular, and more accessible than ever before. And in the centuries since Ptolemy's death, there had been numerous technological advancements that made it easier to put his cartographic ideas into practice. Of course, books were still incredibly expensive during this time, and so atlases became status symbols. People would show off their wealth by purchasing and displaying elaborate maps. These maps had to be colored in by hand, and once cartographers realized that they could charge more if they incorporated more colors, they began looking for ways to do so. The result was the Early Modern borders that you describe, indicated by color, which, while often based on historical provinces or regions, were not intended to convey geographic facts — they were just for decoration. For some examples, take a look at maps by Abraham Ortelius and Gerardus Mercator.

Over time, these artistic conventions made their way into the public consciousness. By the 17th-18th centuries, when rulers looked at maps, they simply expected to see their countries defined by demarcated borders. Additionally, further technological advancements and the development of the modern-day "state" meant that these rulers wanted to — and now could — take stock of their territory. Perhaps the best example of this is the "Cassini survey," which began the 1660s and continued through the 1740s. Louis XIV of France commissioned Giovanni Domenico Cassini to conduct a complete geographical survey of his kingdom, a decades-long project during which Cassini and his team of surveyors went out and attempted to measure and map France using techniques of triangulation. The Cassini survey was considered a revolutionary project, and it inspired other world leaders to undertake similar endeavors in their own countries, the results of which are the later maps you describe.

For more information, I'd recommend looking at Norman J.W. Thrower's Maps and Civilization; he provides a fairly straightforward account of the history of cartography. I'd also recommend looking at work by J.B. Harley, who posits some interesting theoretical ways to understand maps.

EDIT: Formatting

To add something small to /u/ajconbrio's fantastic post, it might be worth noting that when you look at these very old maps compared to modern ones it isn't always useful to think of ours as more "accurate" than theirs. Throughout history maps have been used for many different things, not just representing the layout of political units in geographic space. Certainly this map which you linked earlier might be an accurate reflection of how European physical space was divided between reigning authorities, but that's only useful if that's what you're wanting to show. This famous map, by contrast, is completely useless if you're hoping to get visibility around how the political landscape of the medieval world was structured, but it's not useless if you are, for example, trying to show how the world 'fits together' within a certain socio-religious framework, placing a narrative of Christian history within a geographic framework: it's crowned by a figure of Christ in majesty over the entire world; it has the garden of Eden set in the far east; it marks the tower of Babel and the trails of the children of Israel leaving Egypt. It also depicts the strangeness of the wider world: there's a faceless man in Africa, a man with one very large foot in India, and dog-headed men in far northern Scandinavia.

This is not because medieval cartographers weren't good at making "accurate" maps: this chart is roughly contemporaneous with the Hereford mappaemundi. It's because maps like the mappaemundi represent something entirely different to how we tend to use maps. Rather than trying to accurately set down the specifics of how political units fit together on physical geography, the mappaemundi sought to set down how the ordinary world was embedded in the less ordinary world of Classical and Biblical knowledge. "This is the world you know," it says, "And this is the world of Christianity. See how they fit together, one within the other. See the strangeness of foreign parts." We might think of it as a map of the medieval imagination.

I can't really speak to the specifics of why early modern map-makers would privilege Roman provinces or broad geographical regions over independent political units, except to note that it's widely considered (I understand) that modern states as we think of them didn't really coalesce until after 1648. I think it's important to bear in mind, though, that quite possibly the maps you're looking at aren't wrong or inaccurate, they're just mapping something other than what you're used to seeing mapped. It might be useful to look at the divisions they have placed, and think about what's divided from what. On all of your example maps, France is quite clearly divided from the lands of the Holy Roman Emperor, but the lands nominally ruled by the Emperor are quite often grouped together. What that might mean exactly I don't know, but I'm sure there are people on this sub who do.

There's a book called Vermeer's Hat by Timothy Brook which is tangentially related to this subject, which may interest you. Brook looks at Vermeer's paintings and considers the objects which appear in them—not as subjects of the paintings, but as things which have absent-mindedly slipped into the background because they're part of the artist's ordinary world—as a way of getting at the ordinary world of people in the past. It's a similar kind of approach to using maps as a way of mining how-people-saw-their-world.

I hope this may be useful.

Edit: formatting

Could you link examples of the maps you're thinking of? It sounds interesting, and might help with obtaining answers.

This discussion is so intriguing. I am grateful for the insight that the old “here be dragons” maps are not quaintly inaccurate or ignorant but rather portrayals of a very different way of viewing the world. That’s a paradigm shift. It’s worth pointing out that the focus has been on European history. If maps are representations of how we see the world and guides to help us navigate then it would be interesting to see the evolution of maps in other cultures.