Disclaimer: Steppe societies are not well studied due to a lack of sources and a lack of scholarship, so much of what I am about to elaborate on can be considered conjecture, controversial, or debatable.

Steppe societies are not widely considered to have been states, depending on the definition. Ghenghis Khan and the Mongols are usually cited as an exception, but in reality it was the Yuan Dynasty of China that is the exception, as the Mongols themselves were still built on a traditional model that went back more than a thousand years.

The earliest known organized Steppe state structures are recorded by Herodotus, who tells of the Cimmerians and the Bosphorans, and compares their society to the kings of Egypt. Herodotus however wrote with an agenda, obvious bias, and a style of writing that incorporated both Greek mythology, propaganda, and stylistic editing of detail, which as a result makes his information unreliable. However, what detail does survive suggests that the earliest models of organized Steppe Society can be found in the Scythians, who may have had a tripartite kingdom, where the one king was the high king over the other two. Another important Iranic speaking people were the Kushans, who would conquer the Greco-Bactrians and rule in Sogdia and Baktria for centuries.

The Scythians spanned most of Eurasia, not because they were a single united empire, but because the term Scythian was a generic term (alongside Saka) for the Iranic-speaking nomads that had once dominated the steppes from Manchuria to Hungary. They were united by their language family, but that was all, and they obviously had wide ranges of dialects and their own individual political groupings, organizations, and confederations. However they don't often get the credit they deserve - the earliest known examples of armor and the domestication of the horse can be attributed to these pre-Scythian Iranic-speaking nomads. But I digress.

The first real, highly organized, Steppe Superpower was the Xiongnu confederation. The Xiongnu (pronounced "hong-na", not "hsiung-nu" which is Mandarin) might have belonged to the Yeniseian language family, specifically a dialect called Kettic, but eventually became Oghur-turkic speakers (this happened to the later Mongol Chagatai Khanate as well, and both Turkic and Mongol inherited many Yeniseian loanwords as a result of the early Xiongnu). The Xiongnu developed a state model which would become the standard for every steppe society after them. General history textbooks attribute the dualist structure and decimal organization oftentimes to Ghenghis Khan, but in reality these seem to predate him by 1400 years to the Xiongnu, and possibly were older, we don't actually know. Steppe society didn't develop written records until much, much later than settled culture so much of its development and history is lost. But the Xiongnu, in 205 BC, were united under "Modu Chanyu", Chanyu being their equivalent to "Huangdi" or "High Emperor," basically. In the Xiongnu model they possessed a dualist state structure, which was based facing the South (like the Chinese), in which the Emperor was the High King of the left (East) and his chosen son as heir the king of the East, while there was also the High King of the right (West), who operated semi-independently but was ruled by the High King of the left. The Emperor ruled over the central territory and the tribal meeting ground, where the peoples of left and right came to meet. Under the Kings of left and right were also a King of North and South, so each half of the dualist state was also itself dualist. Adjoining the High Emperor was also a council of "six horns" or "six nobles" which was usually made of close relatives serving as advisors to the Chanyu.

Most steppe states would adopt this model - the Xianbei ("Sebri," or "Serbi," not "Hsien-pei"), Rouran, Hua, and most obviouosly the early Turks ("Ashihna" or "Kok", becoming "Gokturk"). This brings me to my next point: the two great steppe empires before Ghenghis Khan.

Ghenghis Khan was not the first to conquer most of Eurasia, although he did conquer areas these two empires did not (at the same time, at least, they would later after loosing different territories and emigrating), notably China, Persia, and Europe were not held at the same time as Central Asia by the same Steppe polity. Modu Khan was the first, and the Xiongnu Empire extended from Manchuria as far as modern Turkmenistan (where a Xiongnu army was defeated, but that's not important). They even nearly vassalized the Han Dynasty, but over time would eventually be pushed back by the Han. The Xiongnu eventually split into two halves - labelled North and South but probably actually left and right, and the Southern Xiongnu would be assimilated into the Han Dynasty, while various things would happen to the northern Xiongnu. Some would remain, sacking Luoyang in 311 AD, others would help form other peoples, like the Rouran and the early Turks (they were Oghur proto-Turkic speakers by this point), but the majority would emigrate west into Baktria, before drought forced them into Europe where they became known as the somewhat more infamous Huns. Whether the Huns were the same ruling royal line of the Xiongnu is unknown, but the evidence suggests there are archaeological, cultural, and linguistic connections.

This set a trend for inner Asia that is arguably a good reason why one state did not conquer and unite all of it - a constant cycle of steppe polities that formed in the Altai mountains (and later Manchuria) and then emmigrated Westwards due to political and environmental pressures. The Huns were the first, quickly followed by the various Oghur Turkic speakers they had assimialted (the Tiele/Gaoche or Dingling Confederation) who called themselves "Bulgar" as well as "Hun", the Xianbei themselves (known as the Sabirs to the Romans) and then the Avars who appear to have been a splinter of the Hepthaltites, a people called the Hua who broke away from the Rouran (The infamous Mulan is called Hua Mulan, which gives you a hint as to the true cultural influence that built the character. She was not Chinese, and she did not fight the Huns.) The Rouran were powerful, but not as powerful as the Xiongnu, but they did invent the title "Qagan" (Khan). After the Rouran collapsed one of their vassal peoples, the Ahshihna, formed the Gokturk confederation, and the first people who identified as Turks spread out and conquered as the Xiongnu did before. The Gokturk confederation in the 580's AD reached from Manchuria to the fringes of Roman knowledge on the Volga river above the Caspian, and were the people that probably drove the Avars into Europe. This was the era of transition where the Iranic speaking dominated steppes were finally overrun by Turkic-speakers, and Mongolia transitioned from Turkic speaking to Mongolic-speaking (proto-Mongolic speakers had been around, such as the Xianbei). And eventually Tungusic Speakers like the Jurchens coming out of Manchuria would begin playing an important role in Mongolia. That still predates Ghenghis Khan, and they would continue the cycle of Westward movement that characterized the inner Asian steppes.

So as a whole, the Westward movement, barring one major interruption with the early Turks, was a big factor as to why Eurasia had not been united before Ghenghis Khan. Ghenghis Khan could also be said to be a major interruption in this cycle, since the Mongolic speakers would be pushed west as well by the expansion of the Manchus.

(My knowledge is Hunnish-era, not Mongol era, so feel free to chime in on some of this later stuff if anyone is reading).

I hope this helps.

Sources (see comment, too long of a post):