I've seen a couple of maps of the Kingdom of France across the middle ages and noticed that sometimes counties are shown on their own, and that lead me to question something I used to believe, that a duchy breaks up into counties ruled by counts who are in vassals to the duke who in turn is a vassal to the king.
I've investigated a bit and found that at least in England, dukedoms weren't established until the 14th century as more of an aristocratic rank for the sons of the king and that counts pledged their loyalty to the king directly.
So in a general sense, without taking into consideration the exceptions to the rule and focusing more on regions that weren't England, like France, the HRE or Spain, did the hierarchy usually went like king>duke>count? Was a count usually a vassal to a duke?
And now considering the exceptions seen in history, Did counties sometimes become sovereign territories pledging their loyalty to the king directly? Did they remain as counties or Did they turn into duchies eventually?
There was a pretty wide array of potential arrangements that existed in the middle ages, which is also a long period of time. I will stick to Latin Christian states for simplicity, but range pretty far geographically.
Here are some counties that were not part of a duchy or vassals of a duke:
County of Edessa: This was a mostly independent Crusader State in the Middle East, which was sometimes kinda a vassal state of the Kingdom of Jerusalem, but was never part of a duchy or vassal to any duke.
County Palatine of the Rhine: This was a territory along the Rhine in the Kingdom of Germany and Holy Roman Empire. It wasn't subject to a duke or duchy, and in fact the Count Palatine was also the Elector Palatine, which means he was one of the handful of people who got to choose who became the Holy Roman Emperor. There are other counties in the HRE that weren't subject to a duke, but none so prominent.
County of Toulouse: This territory in Southern France was a relatively independent and culturally distinct vassal to the King of France, until it was crushed by the King of France and his Northern French vassals for tolerating a religious minority.
The issue is that, generally, prior to the notions of sovereignty developed in and around the sixteenth and seventeenth centuries (and the Treaty of Westphalia is taken as an easy marker for this emergence of "Westphalian Sovereignty," even though these notions have very deep roots in what is commonly classified as "Enlightenment Thought") the conceptions of state, government, power, and social contract present in Western Europe is completely different from that which we modern observers might have.
And it is the baggage of this mindset which weighed on the scholars who cape up with the neat little feudal pyramid you mention in your post, which you quite rightly question as it doesn't actually hold up to close scrutiny. Indeed, you are not the only person to have reached such a conclusion.
The feudal pyramid which we might be taught to apply to government and authority in Medieval Europe looks suspiciously like the modern administration of an European country like France, Germany, or Italy, with Duchies representing first-tier subdivisions (conveniently similar to modern German Lander or Italian Regioni) and counties representing second tier subdivisions (like French Departments or Italian Provincie). The reality, alas, is much more nuanced. We must expunge any notion of countries, borders, and indeed neat hierarchies of law and authority as we might conceive them in a modern context, and instead reason in terms of what actually manifested. We must accept that there were overlapping systems of authority: aristocrats with their privileges and prerogatives; towns with their courts, guilds, and charters; the clergy, with their monasteries, estates, and fluctuating role in public administration. Feudal subdivisions were thus not miniature autocracies, where the feudatory exerted absolute authority. A feudatory could certainly expand his influence by establishing relationships of dependency and occasionally outright domination, but a feudatory could also be helpless outside of his own lands.
Thus when trying to observe and understand pre-Westphalian society and politics, we must look at the unique structures, formal, informal, idealized, and effective, through which authority and power manifested itself. The story of Burgundy and Aquitaine is different from the story of Yorkshire, as it is different from the story of Savoy and Lombardy.
We can certainly find evidence that a feudal pyramid much like that which you described existed in the minds of at least some contemporary Medieval Europeans. Indeed, it cannot have not existed at all, as this idealized power dynamic and the associated titles would retain their 'ranks' even after the aristocracy's political and administrative role was eliminated. But it is imperative to understand that we are dealing with a world where written laws are slow to change, transmission of these laws is often uneven and dependent on how they can be physically imposed by center of power. Borders, such as they could be said to exist, are physical and social barriers to authority. Local institutions could have complicated and contentious relationships with authorities purporting to exist above them. While we can certainly find examples of treaties marking clear borders between two political authorities, as we can find legal contracts establishing clear relationships between higher and lower systems of authority, it is telling more commonly we find clear physical and practical barriers to power projection: a mountain range, the sea, or city or urbanized region whose own institutions limited a polity from transmitting its authority.
Thus while in this idealized world the King divides his Kingdom into Duchies, in practice these duchies only with difficulty would have clear cut borders, and the Duke's power not necessarily be clearly defined. This could mean thatm in the evolving and changing dynamic between monarchy and regional aristocracy, there were counties with no Duke "between" the Count and King.
Are there instances of Counties being elevated to Duchies? Yes. Are there instances of counties becoming sovereign? Also yes. But diving any deeper would be a study of anecdotes, for there is no systemic answer to easily classify an individual Count's process of power accumulation to obtain autonomy. Indeed if the modern world and the modern understanding of states and state building gives us tools to clearly understand what constitutes sovereignty, these notions were much more nebulous in the pre-Westphalian context. In this period, a hereditary monarchic polity is sovereign because it is able to act as such and it stops being sovereign when it is no longer able to act as such.
I understand this answer is ultimately inconclusive and might be unsatisfying. But I am trying to say there is no real and definite answer, because we need to look at the question a little differently. Did the hierarchy usually go King, Duke, and Count? Yes, at least in theory. In practice, the reality was much more nuanced.