The only group that had a consistent and coherent argument against eugenics of all sorts in the early 20th century was the Catholic Church, because they associated it with birth control, which they opposed in all forms. (I always find this somewhat amusing to point out, because for a lot of people who dislike eugenics today, it is a "right answer, wrong reason" sort of situation.)

Other than that, the idea that "it would be great to eliminate disease and social ills through control of human reproduction" was generally popular among a wide range of people of very different beliefs, including socialists and capitalists, white supremacists and Black activists, feminists and anti-feminists, scientists and demagogues, and so on.

The differences that one finds is how to achieve that, and these groups did differ strongly in their policy imaginations and advocacies. There are big differences, for example, in who advocated for voluntary policies (birth control, in essence) and coercive ones (forced sterilization). Misgivings about coercive policies, along both scientific and ethical lines, ultimately led to a lot of groups eventually breaking away from eugenics as a framework, even if they ultimately still endorsed some of the underlying concepts (the discipline of "genetic counseling" was founded deliberately as a way to avoid the association of coercive eugenics but push for essentially the same biological ends).