According to this list, and other similar lists, the majority of the top universities in the U.S. seem to be private schools, but of the few top-rated public universities, you have either schools in major metropolitan areas in big states (e.g., California, UCLA) or schools with an ancient pedigree (Virginia, North Carolina). Michigan, by contrast, seems to be an anomaly. It is located in a relatively small town in a state that is neither particularly populous, wealthy, or ancient (by U.S. standards). What led to the University of Michigan becoming such a force in research and academia?
Just to note: Ann Arbor, while relatively small, is still a city of over 115,000 people. It's also only 30-45 minutes away from Detroit, a large and economically important city for a good chunk of the 20th century. It's also fairly old, being founded in 1817 (moving to Ann Arbor from Detroit in 1837). North Carolina for comparison, is only about 25 years older.
http://alumni.umich.edu/about/university-of-michigan-history
Okay, cards on the table, I don’t work at U of M, but I work at another one of those so called public ivies, and a land grant school of comparable size and ranking, so this answer is mostly based on my knowledge of the history of another school, but it should give you insight into why there’s some very highly ranked public schools in the US.
The key historical element here is something called the Morrill Land Grant Act, which gave away free land in exchange for building something at the time called “industrial colleges,” a name which they dropped pretty quickly, but it gives you a clue to how people viewed these colleges initially. U of M was the very first land grant college, so it’s the oldest, which may explain some of it’s success right there. These land-grants can be sort of be seen a proto community college: they were expressly for the purpose of teaching practical skills and not traditional college materials. So they have programs in chemistry, architecture, engineering, railroads, agriculture, home economics, really practical stuff. Eventually they will get medicine, law, and liberal arts, but they mostly start off very cut and dry practical schooling for the children of farmers who are looking to better themselves.
Here’s another element: land grant schools had to let in non-white people as a condition of their founding. So these schools are one of the few places where people of color can get a higher education at the turn of the century. So you’ve got some of the best and brightest of a few generations coming to land-grants at this time, simply because they were the only real alternative to the HBCs. They also have to let in women, which has similar effects. I wouldn’t say that students of color had a really good time on campus at land-grants, as they had a lot of discrimination, but they were officially allowed to be students at the very least. You’ll see a few “first black Phd in somethingsomethings” that were degreed at a land grant instead of a HBC.
Add in a few changes in the economy where these practical degrees start to be valued more than the traditional curriculum, and gradually state schools start to come out ahead. You’ll notice a lot of these public schools are very highly ranked in some area linked to their original founding purpose, like engineering for Purdue or UIUC, Ag Science for Iowa, while they aren’t as well known for other programs in the liberal arts area.