Put simply, the salience of cultural issues. Cultural issues were not an issue for farmers then the way they are today.

Before getting into that, we have to clarify three terms which your question runs together. Progressivism, socialism, and populism were very different and had different roots and political bases. Today, they sometimes get lumped together as being generally on the political left, but this obscures important differences that must be understood to answer your question.

The Populist or People's Party was organized around the economic distress of rural farmers who felt exploited by railroads, banks, and a political system more concerned to protect the explosively growing industrial economy than the relatively sleepy agricultural sector. Populists originated much of the language used today to rail against wealthy special interests with corrupt influence over politicians.

For them, these interests weren't primarily the rich so much as railroads, which would exploit their natural monopoly on local transport to extort farmers to bring their produce to market, and Eastern bankers, whose policies, such as the gold standard, made it hard for farmers to get credit (which they needed to buy seed to plant every season). The populists wanted the government to regulate or seize ownership of the railroads and wanted to break the centralized control of the financial system to free up credit, such as through bimetallism. They were also in favor of political reforms such as the direct election of Senators because they wanted to make representatives more accountable to the people.

When you say that farmers supported "socialism and progressive populism," what that really meant was, in the first place, populism. Of the three movements, populism gained a mass following first (peaking in the 1890s), and would serve as the seedbed for farmer political activity through the New Deal. Though the political mobilization behind a left agenda that this original movement sparked explains some farmers later supported the Progressives and Socialists, these latter movements also had very different political bases and aims.

The Progressivism was in the main a middle class movement. It aimed, above all, at the reform of government by purging it of corruption and making it more directly accountable to the people, as well as more professional and scientific. It also sought to regulate capitalism, but not to abolish it. The issue of reform linked the populists and the Progressives, but the Progressives were much more concerned with urban governance and industrial issues like safety and sanitation than were rural farmers.

The Socialists had their main support in urban centers, among the industrial working class. There were substantial efforts at cross-worker solidarity, between farmers and industrial workers, but these never entirely coalesced. The attraction of socialism to farmers is not hard to grasp; the populists' desire for government ownership of railroads and other agricultural services, among other farmer-supportive policies, were entirely consistent with socialist ideology. Scared of big government, farmers were not. But farmers were less invested in some of the socialist movement's central efforts, which were oriented toward confrontations with capitalist factory owners in strikes and efforts at collective bargaining.

Farmers were a mainstay of FDR's New Deal, which included massive government action to support farmers financially and to improve agricultural infrastructure, in the spirit of, if not entirely along the lines advocated by, the Populist Party.

So when did this all change? Like so much of American politics, in the 1960s, with the Civil Rights movement. The Civil Rights movement kicked off what political scientists call a realignment of the parties and their coalitions. The reason is that the Democratic Party became the party of civil rights for black Americans. Before this time, a lot of farmers were Democrats, as part of the New Deal coalition, and a lot of these farmers were Southerners. The national party's backing of civil rights alienated many white Southerners, including farmers. Race, which had always been important but usually not in determining elections, became electorally explosive. Where people stood on race would come to be an important part of where you stood, and who you stood with, in politics overall. This is the origin of what we today often euphemistically call "cultural issues."

In the years following civil rights, a new politics would emerge, midwifed by people like Nixon and Reagan, which divided Americans along cultural lines. Were you on the side of the forces that were upsetting traditional American life? The dope-smoking, free-love hippies? The frightening and ungrateful Black Panthers? The student radicals disrespecting the troops and the flag? Did you support black advancement, even if it meant affirmative action? Did you support women's liberation? New sexual mores? Divorce? Abortion? All these new "cultural" questions served to displace the economic dimension of political competition which had defined the Populist era. Now, farmers found themselves on the opposite side of these questions from those who supported left policies.

And that brings us to today. Farmers are today a very small group, but they are generally heavily rural, and rural Americans are culturally identified with the party that opposes most of the things the Populist, Progressive, and Socialist parties stood for. This identification was the outcome of the civil rights realignment, which raised the salience of race as an issue and made possible its exploitation to divide Americans along cultural lines.

Sources:

Hofstadter, The Age of Reform

Goebel, A Government by the People: Direct Democracy in America, 1890-1940

McCarty, Poole, and Rosenthal, Polarized America

I am not a historian, but I am a social scientist working in critical agrarian studies (an interdisciplinary field), so I can offer you some well-researched explanations on changes in agrarian social structures around the world and in the US that help answer your question. I am not able, however, to go equally in depth into elements pertaining to the specific history of US politics that make up another part of the answer to the question (eg repression of socialist movements and workers' rebellions along the twentieth century, changes in religion, etc.), which I'll only briefly allude to, so I hope others can complement on that.

I assume from your question and the references to Eugene Debs and William Jenkings that you are asking about farmers supporting progressive politics in the United States, specifically in the 20th century, although you didn't specifically say so. I should state, however, for context, that the potential "progressive" political role of landowners and peasants is a classical question in social science - especially within the history of Marxism, where in more orthodox versions the working class is typically presented as the revolutionary class and the peasantry is considered either an outdated class (remnants of feudalism) or as having interests closer aligned with the bourgeoisie and with larger landowners (since they often also own or detain land, differently from those that only own their labor-force and thus "have nothing to lose"). But there have also been varying streams within Marxism, especially in the 20th century, recognizing the peasantry as also being in conflict with capitalism, being constantly vulnerable to being dispossessed, indebted and also having their surplus labor robbed, and therefore often having a key historical role constructing radical politics around the world (eg authors Theodor Shanin, Eric Wolf, a large part of Latin-American Marxism, Maoism, etc.). Outside the US, the "rural poor" (a varied social segment, including peasants, rural workers, small farmers, indigenous peoples, among others) have often continued to have a more progressive role in politics, and a classical book on this is Eric Wolf's "Peasant Wars of the Twentieth Century".

You can note that I was using the word "peasant", which is a word often used to describe small farmers in medieval Europe and also in the contemporary Global South. But this word has less typically been used in the context of the United States, which has its own history of colonial settlement, wider massacre and less miscigenation with indigenous peoples (compared to Latin America), strong imaginary of conquest of the "frontier", in which "farmers" have played a key role, and the influence of protestant ethics. This all has led us to associate "farmers" in the US, even before the twentieth century, with entrepreneurship, self-sacrifice, individual risk-taking and economic units centered around the family instead of the wider community, which are all elements that can often be swayed towards regressive politics instead of progressive politics, depending on the historical moment.

When we talk of the history of the politics of "farmers" in the US - do we refer to the tobacco plantation owners in the eighteenth century? To the small farmers escaping the Dust Bowl? To emancipated slaves trying to access land to cultivate? As you can surmise, the word "farmer" can mean many different things, in different historical contexts, with different implications of class, race, family formation and gender relations. In the case of the United States, agrarian structures have suffered incredibly drastic changes in the twentieth century due to economic concentration and technological change of capitalist agriculture, to the point where we can say now that most of the smaller or mid-size farmers - the ones you ascribe to supporting socialist politics in the beginning of the century - have simply been extinguished. Tony Weis points out the starking figure that, at the end of the twentieth century, there were more prison inmates in the United States than there are full-time farmers - that is, US full-time farmers now make up less than 1% of the population.

He writes: "A few snapshots of the US farm economy at the end of the twentieth century indicate the magnitude of this transformation: the area of cultivated land per agricultural worker in the US (58 hectares) is roughly fifty times the world average (Gilland 2002); farms greater than 500 acres (203 hectares) control 79 per cent of all US farmland; a mere 8 per cent of farms (with sales of at least US$250,000) account for 72 per cent of all farm sales along with the majority of farm subsidies; and the population calling farming their principal occupation was under 1 million, with the bulk of production coming from only about one-third of these (USCB 1999). There are more prisoners in the USA than full-time farmers. The flipside of the massive industrial operations that provide the bulk of production are the much smaller-scale family farms barely hanging on." (Weis, 2007, p. 83).

Throughout the twentieth century, in the US and in many other countries, agriculture and livestock raising became much more technified and industrial, with use of Green Revolution technologies (and later biotechnologies), requiring farmers to have more and more capital each season merely to get by in increasingly competitive industries. Besides the competition among them and the transformation of "family farmers", the food commodity chains as a whole became increasingly corporate-owned and oligopolized, from the industry of inputs and machinery, to trading, transport and retail, strangling smaller farmers from all sides of the value chain. Food production also became concentrated in a few key commodities, used as flexible ingredients for a variety of processed foods (as well as fuel, industrial use, etc): corn, wheat, soybean, and animal meat (which exploded in the past decades and required more and more grain and oilseeds for animal feed), and there was an increased division of labor and global trade of food, with much of American produce becoming imported from abroad.

Of course, there has also been resistance to this and even efforts of small farmer revival, as can be seen today in the proliferation of farmers' markets, Community-Supported Agriculture, urban gardens, etc. The National Family Farm Coalition is an example of ongoing farmer association with progressive politics, which mobilizes "family farmers, fishers, and ranchers for fair prices, vibrant communities, and healthy foods free of corporate domination" (their website). We can also look at more recent efforts to mobilize migrant farm workers, which in many senses might be closer in class interests to small farmers at the beginning of the twentieth century than some of the large soy producers now (even if they can also be strangled by the transnational agribusiness corps and trade and taxation policy).

Concurrent to those economic changes, authors have recently called attention that regressive politics - or right-wing populism - was already spreading among farmers since the 1930s and especially since the second half of the twentieth century, often in a coordinated reaction to the labor and socialist movements that had previously gained force. Montenegro de Wit et al. (2019) contrast the historical spread of regressive populism in California and in the Midwest. California agriculture in a sense was already born more "capitalist" and developed later in US history, so emerging larger farmers could more easily use authoritarian populist tactics against organized farm labor and otherized migrant and racialized minorities. In the Midwest, meanwhile, farmers had historically been more disenfranchised and politics fractured more across class lines. There too, however, there were significant backlashes since the 1940s, with the economic context described above of consolidation of larger agribusiness, as well as Cold War-style repression of labor and left-wing movements. The authors point out that, even in the 1980s, however, the Midwest saw the rise of new left-wing and more radical farmer movements in reaction to falling prices - although a part of them also adopted racist discourse (we should note that, historically, there have been often awkward connections between defending disenfranchised and impoverished communities while still otherizing internal or external racial minorities, which some might see as simultaneously "progressive" and "regressive" movements, depending on how you classify these terms).

(continuation and sources in comment below)

This is a complex question with hundreds of books written about it. It’s always hard to say why something didn’t happen, and it’s also always hard to write a simple account of such macro-sociological phenomenon.

So, this cannot possibly be an exhaustive list, but let me now just run through some of the more popular answers:

America’s Unique Class Structure The first is the classical Marxist explanation. First thing we have to realize is that most people in most countries were rural (later note: I spent a little too long focusing on rural politics because I misread your question, but I promise part three gets into explanations). There, the rural peasantry/serfs and rural proletariat are inherently “non-revolutionary”. Let me point out that, before you even gave all American examples, the way you phrased your question clearly put it in the (North) American context. When we see Left Wing movements of various kinds in Europe, the rural population is frequently either divided, a non-factor, or on the side of the reactionary right. The most famous study on this I know is Charles Tilly’s The Vendée. See, the Vendée was a region in France that witnessed the most significant popular uprising against the French Revolution. Their state purpose was that they were defending the King and the Church (pay attention to the (c)hurch, because that’s going to be a reoccurring theme). To overly simply the book’s nuanced argument, Tilly essentially argues that the peasant rebels in the Vendée felt that they had more in common with their noble landlords than they felt they did with the distant revolutionary democrats of Paris. This sort of thing is common. In 19th century Russia, Narodist (non-Marxist social movements) kept “going to the peasants” and the peasants kept… not really doing much for their political program. Certainly, the peasants might occasionally sack the manor house and burn all the records of debt, but peasant agitation (as James C. Scott has argued in basically all of his books) is very different from the urban social and political movements we imagine. The peasants resisted systems of control through “every day resistance” like “foot-dragging, evasion, false compliance, pilfering, feigned ignorance, slander and sabotage.” But the peasants’ concerns were mainly local.

Marx sort of hated them for this reason. Well, hated them politically, at least. Here’s a little bit from the *The Eighteenth Brumaire *:

The small-holding peasants form an enormous mass whose members live in similar conditions but without entering into manifold relations with each other. Their mode of production isolates them from one another instead of bringing them into mutual intercourse. […] A small holding, the peasant and his family; beside it another small holding, another peasant and another family. A few score of these constitute a village, and a few score villages constitute a department. Thus the great mass of the French nation is formed by the simple addition of homologous magnitudes, much as potatoes in a sack form a sack of potatoes. Insofar as millions of families live under conditions of existence that separate their mode of life, their interests, and their culture from those of the other classes, and put them in hostile opposition to the latter, they form a class. Insofar as there is merely a local interconnection among these small-holding peasants, and the identity of their interests forms no community, no national bond, and no political organization among them, they do not constitute a class. They are therefore incapable of asserting their class interest in their own name, whether through a parliament or a convention. They cannot represent themselves, they must be represented.

So, what you’re seeing in your American history textbooks is in many ways atypical of the political behavior of rural populations globally. I tend to agree to some extent with the right-leaning political scientist Samuel Popkin who argued in his seminal The Rational Peasant (the book is an answer and “rebuttal” to James C. Scott, but I think they can be read side by side just as easily as against each other) that the peasant is in many ways inherently conservative because both long and short term substance crises are always on the horizon. The peasant is small-c conservative because change always has the risk of increasing the odds of famine.

But America doesn’t have peasants, and that’s a hugely important distinction. The quintessential American for much of the 18th and 19th century (whether we follow Jackson or Jefferson’s vision, Crevecoeur or de Tocqueville’s) was the yeoman farmer. There were, of course, many other types of rural labor relationships, especially slavery, sharecropping, indentured servitude, and rural proletarians (that just means they worked for someone for wages, rather than primarily working their own land), but the one seen as most typical—and the one common to all the examples you cite—is the yeoman farmer. Yeomans owned land, peasants frequently didn’t (they were tenants of large landowners, or worked as wage earners). Peasants frequently had a relationship with a lord or local notable. The yeoman farmer was meant to be an equal citizen (at least in America) and in many communities the class inequality wasn’t particularly severe, at least compared to Europe. Even in England in this period, the rural class structure is frequently thought of as “landlord, tenant, and laborer”, with the freeholding yeoman relatively rare. But in America (and in many other settler colonies like Canada and Australia), they were the norm.

The term “American exceptionalism” has come to mean in the present day some sort of jingoism that America cannot err and has a unique, quasi-divine role on the world stage, but the first recorded instance of the phrase is actually not this sort of thing at all. Stalin, apparently, was the first one to use the term arguing against American exceptionalism. What was the exception? Well, some members of the Communist Party were arguing that America didn’t quite fit into the classical Marxist structure because American class structure was fundamentally different, in large part because of these yeoman farmers. (How connected the usages of the 1930’s in Communist circles and the current use by the Neoconservative Right is argued over; the tern’s Wikipedia page has a decent run-down).

Obviously, later Marxist thinkers starting really with Mao argued that the peasants could be revolutionary, and there were attempts at Marxist peasant revolutions in Latin America, South East Asia, and Sub-Saharan African. All Marxists count peasants as "workers", and one of the key constituent group of socialist society (fun note: the hammer and the sickle is meant to represent the urban worker and the rural peasant, respectively. Some communist movements have updated those symbols to be more meaningful to the local time and place. The best is probably the flag of Angola, which contains a gear and a machete, representing the tools that Angolan urban workers and peasants were commonly making use of in 1975). We have seen peasant-led revolutions, but usually one of the most fundamental claims is land reform (i.e. redistribution of land from the big land owners to the peasants). While it's rarely discussed in America, this in many places was the fundamental grievance for rural politics and also the biggest wealth transfer in national history. America even at one point forced the South Vietnamese government to perform land reform in hopes of stymying support for the Viet Kong.

But that, in many ways, only emphasizes American exceptionalism, that likely also applies to Canada, Australia, New Zealand, etc. Rural class relations, and therefore rural politics, was so fundamentally different in America that an explanation dealing with America couldn't possibly apply to many other parts of the world. Even in other parts of the New World (not to mention much of the American Deep South), colonial powers created the hacienda and similar large land-owning estates created a creole aristocracy who either had slaves or peasants working the estates, essentially recreating the same sort of hierarchical rural social relations that long dominated much of the Old World.

So, that doesn't explain anything in and of itself, and is really just prelude to what did happen in America. Before we get into the more specific explanations of what happened between now and the point of time you mention, I thought it was important to note here that, in many ways, that period of time for American farmers was the more atypical one, world-historically.

(continued below)

Hello! If you are a first time visitor, welcome!

This thread is trending high right now and getting a lot of attention, but it is important to remember those upvotes represent interest in the question itself, and it can often take time for a good answer to be written. The mission of /r/AskHistorians is to provide users with in-depth and comprehensive responses, and our rules are intended to facilitate that purpose. We remove comments which don't follow them for reasons including unfounded speculation, shallowness, and of course, inaccuracy. Making comments asking about the removed comments simply compounds this issue. So please, before you try your hand at posting, check out the rules, as we don't want to have to warn you further.

Of course, we know that it can be frustrating to come in here from your frontpage or /r/all and see only [removed], but we thank you for your patience. If you want to be reminded to come check back later, or simply find other great content to read while you wait, this thread provides a guide to a number of ways to do so, including the RemindMeBot- Click Here to Subscribe - or our Twitter.

Finally, while we always appreciate feedback, it is unfair to the OP to further derail this thread with META conversation, so if anyone has further questions or concerns, I would ask that they be directed to modmail, or a META thread. Thank you!