Was it ever considered as a serious proposal? Would it really have led to millions of Germans starving?
There's a bit to unpack and analyze not just in the answers to these questions, but in the questions themselves. Let's break some of these down.
What exactly would the Morgenthau Plan have entailed?
To start: the Morgenthau Plan is the name typically given to a briefing prepared by Morgenthau, entitled "Program To Prevent Germany From Starting a World War III" (full text in the link). This was delivered at OCTAGON, a meeting between FDR and Winston Churchill in Quebec in 1944.
The plan is indeed rather radical. It calls to carve up Germany into smaller states, to deindustrialize Germany directly by destroying or taking industrial capital, to never demand reparations (so that industrialization would never be necessary), effectively turning Germany into a collection of rural farming nations.
So now we might want to address "Was it ever considered as a serious proposal?" In some very direct sense, yes. Indeed, as mentioned, this briefing was proposed to Churchill himself at OCTAGON. It had the backing of FDR, and formulated the backbone for JCS 1067 - which would establish U.S. policy towards postwar Germany for the immediate future. However, this proposal was simultaneously not serious, and was never going to realistically be implemented to its fullest extent. To explain this seeming contradiction, we should look more closely at who Henry Morgenthau was, and how the Morgenthau Plan got as far as it did.
Henry Morgenthau was FDR's Secretary of the Treasury from 1934 until his resignation under Truman. Morgenthau was appointed Secretary as a fiscally moderate proponent of the New Deal and as a staunch ally of FDR. Critically, Morgenthau had done work to discredit FDR antagonists in the early years of the New Deal, and over time built up his reputation in the treasury. Critically, as the war broke out and FDR became more concerned with the war at a higher level, he delegated more responsibility to Morgenthau. Morgenthau was able to leverage his good graces with FDR into becoming one of the more powerful men in his cabinet. Lend-lease was a massive factor in the allies' ability to prosecute the war, and Morgenthau served as an advisor to FDR to get lend-lease working despite opposition from isolationists and conservatives within congress. This also gave Morgenthau leeway to conduct foreign policy without much interference, as he now had to meet regularly with British politicians. By the end of the war, Morgenthau was in near total control over international diplomacy about economic concerns - things FDR knew little about anyway.
It's no mystery then how Morgenthau came to have a prominent position in the economic structure of allied-occupied Germany post WWII. And Morgenthau - much more than anyone else in FDR's government, nonetheless the British - had very strong feelings towards what needed to be done to prevent future war. In 1944 he was quoted once as saying to Henry L. Stimson, Secretary of War, "Don't you think the thing to do is to take a leaf from Hitler's book, and completely remove [German] Children from their parents and make them wards of the state, and have ex-US Army officers, English Army officers and Russian Army officers run these schools and have these children learn the true spirit of democracy?" When Stimson later remarked that Morgenthau's plan would involve displacement of millions of Germans, Morgenthau is quoted as saying "Well, that is not nearly as bad as sending them to gas chambers."
Despite Morgenthau's strong feelings on the matter, he faced much resistance throughout FDR's cabinet on the issue. E. F. Penrose, who served as an economic advisor to an ambassador, criticized it as not being an intellectually respectable plan. Stimson criticized it as "clumsy economic action," which would "produce a very dangerous reaction in Germany and probably a new war." Morgenthau's son believed his father had "proceed[ed] with a driving fury, confounding his most ardent and loyal supporters." Nonetheless, when pressed on the matter of how it may effect civilians, Morgenthau is quoted as saying "I can be overruled by the president, but nobody else is going to overrule me." FDR, in contrast to virtually everyone in his cabinet, tended to side with Morgenthau on the issue. Morgenthau recorded FDR as saying "We either have to castrate the German people, or you have got to treat them in such a manner so they can't just go on reproducing people who want to continue in the way they have in the past."
And thus, with Morgenthau's fierce pressure and FDR's graces, Morgenthau pushed the plan on the British at OCTAGON. And, perhaps unsurprisingly, the British found the plan untenable. After proposing the plan, Morgenthau recalls Churchill "turn[ing] loose on me the full flood of his rhetoric, sarcasm, and violence" - insisting the plan was an utter mistake. However, Britain was in dire straights by this point in the war. One important piece of context when considering British talks during WWII was that, when WWII started, Britain was still in debt to the US for WWI - it had very little negotiating power, even in the war's earliest days. This is why lend-lease was even necessary - Britain had no way to pay for American war materials.
Thus, OCTAGON wasn't a meeting of even-handed diplomacy between equals. The United States had unilateral power, and Britain was in desperate need for a new round of lend-lease. Churchill, probably, sensed that fighting a fight against a hypothetical future didn't make sense, compared to the material now, and relinquished the fight, and agreed to the Morgenthau plan. Churchill is quoted as stating afterwards, "[T]he future of my people is at stake, and when I have to choose between my people and the German people, I am going to choose my people."
Stimson, and Cordell Hull, FDR's Secretary of State, later grilled Morgenthau over the meeting at OCTAGON. At this point, the rift Morgenthau had formed by pushing for such a radical plan was beginning to alienate him within the cabinet. Despite this, JCS 1067 managed to win enough votes, and became the document guiding U.S. economic development (or rather, anti-development) in the immediate post-war era. It's important to note that JCS 1067 was not the full Morgenthau plan by any stretch - it was the parts of it recommended to the army, only for the immediate occupation. However, as JCS 1067 was garnering fierce criticism, and the Morgenthau plan's details were becoming known, it was leaked to the press and was published widely.
The public reaction to the plan was swift and severe. Major American newspapers denounced the plan. It served as the most valuable type of propaganda - the kind based in some amount of truth - for Goebbels, who spread news of the plan far and wide throughout Germany. Some American generals - Eisenhower included - believed news of the plan stiffened resistance a substantial amount. It was a PR disaster, and one that Morgenthau would never recover from. For his part, FDR began to immediately distance himself from the plan, lying that he was "frankly staggered" by what it said. FDR also dissolved the cabinet responsibly for JCS 1067 in a public display of disapproval. Churchill would make similar remarks, denouncing the plan. Later, in 1949, Churchill would even apologize for it.
Morgenthau's reputation was so damaged he would remain persona non grata until Truman recommended his resignation in July 1945. JCS 1067 would be rescinded and replaced by JCS 1779 - which called for Germany's economic growth - in 1947, after U.S. foreign policy grew hostile towards Soviet presence in Europe and after believing JCS 1067 had severely hampered German recovery after the war.
Now we can come back to the seeming contradiction above. As should be clear now, The Morgenthau Plan - despite making it very far through negotiations - was primarily due to the influence of Morgenthau himself. However, there was also much larger - and much more fierce - opposition to it, which simply fell to the wayside under the vicissitudes of war. The plan lived and died alongside Morgenthau's influence over the president. And since the plan was driven by one man, it is extremely unlikely that the plan would have been taken so far as to actually fulfill its ends. It was simply too unpopular.
Lastly, as for the question of whether millions really would have died - is impossible to say. Men who analyzed the plan at the time, within the U.S. cabinet - certainly believed it would displace potentially millions of people. Goebbels, in typical war propaganda fashion, amplified fears as far as possible. However, due to the fact that the plan did not go into such depths that it can be rigorously analyzed, nor was it ever a realistic option anyway, it's hard to conclude anything concrete. It's usually impossible to answer such counter-factuals - so much would already have to be different in order for such a plan to happen, that you're entering a realm of total fantasy before an answer is within grasp. Despite this, it seems it would have at least been rather devastating, though likely far below what Nazi propaganda would have you believe.
I pull much of this from "The Battle of Bretton Woods" by Benn Steil - while Morgenthau is more of a secondary character in it, there's a long dedicated section dedicated to this exact topic in it. In addition I have propped up some points with additional reputable sources, such as the links to primary source documents, in this post.
While you’re waiting for more contemporary discussion, u/versager makes an interesting comment on the possible pros and cons of this actually taking place here, though I admit it’s of rather suspicious quality.