In brief: In a globalizing world, millets were forced to compete with and were gradually supplanted by alternative crops and cropping systems, culminating with the rise of industrial agriculture and the focus on a narrow band of favored, high yield crops that did not include millet.

In more words: Millet - really, millets - are a collection of hardy, nutritious small-grain cereal species. There are about fifty different species we call millet. That hardiness is the essential thing to understand about the cereals - millet is a fast growing summer annual which takes about half the time to mature compared to wheat and rice, it's very tolerant to drought, poor and acid soils, and quite disease resistent. Nature and nuture have made millet a crop that's hard to fuck up.

Among the major varieties, foxtail and broomcorn millets originated in or around the hills of the Yellow River valley as early as eighth millennium BCE and no later than the sixth millennium BCE. The cultivation of east-Asian millet varieties intensified and expanded through the third millennium BCE, spreading along the "Inner Asian Mountain Corridor" that would one day become the Silk Road and into Europe by the second millennium BCE.

The major African varieties, pearl millet and sorghum, are found being widely cultivated in their domestic forms in the west and east Sahel zone respectively, by the end of the third millennium - interestingly, these crops spread extremely rapidly to India where they became important staples by 1700 BCE. Among the neolithic peoples of the semi-arid foothills of Eurasia and Africa, millet was an immensely valuable, low risk crop that could be grown with minimal cultivation to supplement a pastoralist or multi-modal subsistence lifestyle. With advances in irrigation that developed over the next thousand years, millet would spread out of the foothills into arid valleys and plains where it would remain an essential staple through the first millennium CE and, in places, on into the second millennium.

In your question, you said millet is best known as birdseed, but it's worth noting that millets still rank among the top 10 most important cereal crops in the world, and they remain staples in much of India and the African Sahel. Still, millet production peaked the 1960s and has disappeared altogether in many regions of the world.

The first part of 'why' is technological, the second part is cultural. I've already mentioned millet's drought resistance in contrast with a crop as thirsty as rice, but also wheat and maize. Now, drought resistance is obviously a great quality if you're worried about water scarcity, but it comes with a downside: under good conditions, millet has historically yielded about a half of what wheat does and a quarter of that of rice (~1:2:4 tons per hectare, respectively). Rice may be much more capital and resource intensive, but those locales which are able to build that agricultural capacity quite literally reap the rewards.

Which locales were those? Outside a few regions of the world, they are the richest, the most centralized, the most powerful - and what we see is that as these alternative cereals (particularly rice) become the food of the rich, millet, by default, becomes the food of the poor. In 1807, Francis Buchanan observed

The crop of Ragy [a millet] is by far the most important of any raised on dry field, and supplies all the lower ranks of society with their common food. Among them, it is reckoned the most wholesome and invigorating food for labouring people; and in every country, most fortunately, a similar prejudice appears to prevail, the most common grain always reckoned the nourishment most fit for the labourer... My Bengal and Madras servants, who have been accustomed to live upon rice, look upon the Ragy as execrable food, and, in fact, would experience great inconvenience were they compelled to live upon it. (Buchanan 1807:102)

So far as I have observed in Mysore, ground, once brought into cultivation for rice, is universally considered as arrived at the highest possible degree of improvement; and all attempts to render it more productive by a succession or crops, or by fallow, would be looked on a proofs of insanity. (Buchanan 1807:93)

As an illustrative example, we see the gradual expansion of cities in southern India, founded by religious and political elites though the middle period who prized rice as a staple crop and embodiment of abundance, who directed the construction and expansion of irrigation systems and agricultural infrastructure into areas where millet had been prevalent.

This exact situation isn't necessarily repeated elsewhere, but we see similar overall dynamics throughout Eurasia. Foxtail and Broomcorn millets have been important crops in the north of the Iberian Peninsula since the approximately 1600 BCE, where they could be planted as a summer crop in rotation with slower growing winter wheat or barley, and it remained important through the middle ages. Galacian tax records from the archbishopic of Tui record payments of millet on a comparible level to wheat, and more frequently than rye, barley, or legumes. This seems to have changed in the early modern period when maize was introduced, and millets were phased out on the less marginal lands. Today, millet porridge is generally regarded as 'poor people's food' in Galacia, and it's mostly disappeared as a food crop elsewhere on the peninsula.

That was the reality going into the 20th century. Enter Norman Borlaug, the Rockefeller Foundation, and the mission to export what Michelle Kaika called "Modernity's Promethean Project" to "backwards counties" across the globe. Whole books can and have been written on the "Green Revolution", but for our purposes, the Green Revolution meant the export of American industrial agricultural practices across the globe. Under this new agricultural regime, high yield, intensively farmed, fertilized, global generalists like dwarf wheats and rice were the order of the day, and they were supported by large scale interventions in land and water infrastructure. With alternative cereal yields increasing by 50, 100, or 150%, and subsidized by the state, millet has largely been sidelined. Today there's some increasing interest in millet and other "indigenous" or "low-impact" crops, but that's a discussion to be had in another forum (or another 20 years hence).