As per the tenets of the feudal system (which were never completely, perfectly, and cleanly applied) a King is the embodiment of the state in a physical person. Because of reasons, European monarchs divided their kingdoms among several Dukes, who ruled great swathes of territory.

In Italy, for example, Charlemagne made his son Pepin King, and subdivided the territory among several duchies, such as Lombardy (granted to the Obertenghi) and Tuscany (granted to the House of Canossa).

The difference between a Duke or a Marquis (Marquis being the Franco-English translation of the German title Magrave) is that a Marquis is the ruler of a March, or border territory. Again, in the example of Italy, Charlemagne established the March of Friuli, and granted it to the House of Ahalolfing to guard against encroachment by Slavic peoples. Marquis' had more liberty in how they administered their lands, as they had greater defensive responsibilities, however, they could come under much more scrutiny. For example, in 828, a diet at Arachen removed Balderic Ahalolfing because he had proven particularly ineffective at repelling Omurtag of Bulgaria (in Balderic's defense, Omurtag was a serious ass-kicker. Balderic didn't much oppose the decision, his family held lands in Germany that were less prestigious, but less of a headache to live in as well).

Each Duke or Marquis in turn subdivided his land among several Counts. For example, again in Friuli, there were four Counties. In more densely populated regions, there were many more (like Lombardy. I've been trying to count them. It's harder than it seems). Although small, independent counties subject only to the Emperor did exist in the Holy Roman Empire, they weren't entirely common, although most counts did exert lots of authority with lots of ways to abuse their power even if they were theoretically subject to a Duke or Marquis who should be keeping them in check. Most counts also had their own vassals, which in the idealized version of feudalism should be called Barons (although in Italy, the title Barone was only ever introduced in the south after the Sicilian Vespers, most small landholders would simply be called Cavaliere, or "Knight").

Of course, Italy soon saw the feudal system break down because of reasons, and while Germany remained the core of the Holy Roman Empire, the union of various duchies along with infighting and continuous internal changing of borders eventually meant titles had little meaning beyond the privileges they bestowed. By the thirty years war and the subsiquent reorganization of Germany, a Magrave ranked the same as a Duke, sometimes rendered as "Prince" (in pop-culture history, a Prince is a King's son, however it more commonly is the title for the ruler of an independent state).

In Italy, independent states were often termed as Grand Duchies (It. Gran Ducati, sing. Gran Ducato) because many, such as Lombardy and Tuscany, had borders that corresponded to the old Carolingan subdivisions, with the key addition of "Grand-" meaning that they were independent.

edit: typos

edit 2: It was pointed out that I wrote "Tenants" instead of "Tenets". Whoops!