I've heard both that "collectivisation never works" and that "it worked in Burkina Faso".
I understand that the famines in the USSR continue to be a controversial and finger-pointing subject, and I don't want to get into ideological warfare here.
I would just like to know, as neutrally as possible, what was the difference between these two situations?
Collectivization did not fail in the Soviet Union, but achieved its intended effect, and the Communist Party of the Soviet Union was relatively unconcerned with humanitarian side-effects in the countryside. The CPSU saw collectivization not just as way of increasing the efficiency of the backwards Russian agricultural production system, but of seizing surplus crops from peasants in order to export them and gain foreign credit to industrialize.
We are fifty or a hundred years behind the advanced countries. We must make good this distance in ten years. Either we do it, or we shall be crushed.
Soviet collectivization was advertised as way of improving efficiency in Soviet agriculture in public - this was basically the only case that could be made to peasants to get them to comply with abandoning the land they had so recently just received from the redistribution. However, its true goal, as expressed in Stalin's party catchetism - his Q&A sessions that he staged in the politburo - was to fund industrialization.
The Soviet Five Year Plans were expensive. Unlike the later DIY approach taken by the Chinese Communist Party in the Great Leap Forward, the Soviet Union built no backyard furnaces. They imported American advisers from Ford Motor Company among other industrial conglomerates for double the pay they would have received in the US. They contracted foreign companies to build modern plants with their own employees, and they imported record amounts of foreign industrial machinery. To pay for all this, the Soviet Union needed to export to gain foreign currency, which required agricultural exports.
Collectivization was advertised as a way of increasing agricultural efficiency, and it partially succeeded and partially failed in this respect. Russia and Ukraine had been plagued by famine for centuries, and the 1932-33 famine was the last, excluding the post-war famine which was fuelled in large part by high quotas left over from wartime mobilization. Despite a massive influx of rural labor moving to the cities, the Soviet Union's agricultural production declined by only small proportions. Until Khrushchev deconstructed the agricultural machine distribution system, there were improvements in production through greater use of modern farming methods and fertilizers.
However, productivity was a secondary goal of collectivization. Collectivization was a war on agricultural surplus - a very prominent word in Marxist thinking. Surplus was the amount of labor after production that a worker in Marxist theory could use to exchange with the marketplace, outside his own subsistence. In capitalist systems, Marx theorized that the vast majority of surplus was taken by capitalists, leaving the proletariat in a permanent state of subsistence without the ability to accumulate savings. However, this surplus, even if it was in the hands of the proletariat, was the basis for the formation of markets and equilibrium pricing, which disrupted Marx's preferred labor theory of value. In his view, value of goods should be determined by man hours dedicated to creating them. While he never directly stated it, the logical conclusion from this argument would be that all hard workers are paid the same, no matter their profession.
During the period of collectivization, traditional Marxist paranoia towards surplus took over. Nowhere, is the his clearer in the dekulakization campaign. Under the preceding liberal period in Soviet economics called the New Economic Policy, farmers could sell their crops and trade in markets on a small scale. This led to some peasants becoming marginally wealthier than others - though no modern Westerner would consider the condition of the so-called Kulaks "wealthy" by any stretch of the imagination. However, even these really quite poor "elites" were made targets by the party, as they were accused of hoarding their surplus. They were made to "voluntarily" double-tax themselves. Party heads in districts were judged on their ability during collectivization to raise large amounts of agricultural production which the party would use to feed the growing cities or export to the West. This meant the Soviet economic mode in the countryside became extractive - one which focused on wringing all surplus out of the peasantry. Collectivization was therefore not a genuine drive towards equality or modernization, but a control mechanism over the peasants.
The Soviet war on all kinds of privately held surplus continued throughout the Stalin period, abating only after the General Secretary's death, and not just in the countryside. Some estimates place the amount of industrial investment as a % of GDP during peacetime in the Soviet Union and pre-1954 Eastern Bloc states at between 25-50% depending on the country. For reference, 6-10% is considered very high in the West. To afford so much investment, personal consumption had to go down. Despite an astronomic increase in the number of industrial workers during the Stalin years, the average wage of a Moscow industrial laborer actually dropped by 19%. Soviet Central Planners preferred this investment-heavy approach for three main reasons. First, ideologically, privately-held surplus, as I detailed above, was anathema. This was especially true in the eyes of planners, who used Marx's labor theory of value as their way of pricing goods in the Soviet Union. To them, the world of small-scale kulak capitalism was alien and impossible to understand. Second, they wanted to leverage a multiplier effect, which was critical to their theory. They believed production of capital (re-investable) goods, such as steel, machinery, etc. would lead to exponential growth. Finally, in the absence of profit motive, Soviet managers developed a power motive and sought to increase their control over resources, either through turf battles with one another, or through appropriating it from private citizens.
A secondary motivation for collectivization was purely ideological. According to Marx, the proletarian wage-earner was the constituent of Communism, but Russia was an agrarian nation. Peasants were seen as prey to religion, superstition, and conservative sentiments which could make them a constituency for counter-revolutionaries. Since the Soviet Union had no proletariat, the CPSU had to create one. This involved expanding cities, but also turning peasants, who had become small business owners after land redistribution and NEP, into employees of the state - many of whom were paid in wages after.
The economic model and ideological justification behind collectivization help to explain the CPSU's callousness towards mass starvation. Any outside eye analyzing the Soviet economic plan could easily have concluded that taking what little surplus an impoverished peasantry had in good years put them in danger when natural disasters hit - which was the case in the early 30s. However, the goal of industrialization was seen as more important, because it was a question of national security and the survival of socialism. Second, the death of the countryside to feed the city was seen as regrettable, but not entirely undesirable by Marxist theory.
I have spent so much time detailing the process of Soviet collectivization in order to prove a potentially controversial point: that, unlike the Burkina Faso collectivization scheme, the Israeli Kibbutz, or other collective farming operations around the world, Soviet collectivization was never meant to help the farmer. It was completely different than all collectivization schemes that were not run by the CPSU and its client states, and all collectivization schemes that did not coincide with massive industrialization. The starvation of millions of people in the early 1930s met an unmoved response from Moscow. The CPSU did not see the humanitarian crisis as necessarily their fault, as Russia and Ukraine had many famines before, and, more importantly, did not see it as a catastrophic failure. Addressing the starvation and problems arising from collectivization, Stalin simply said that economic planners were dizzy with success. Soviet industrialization was phrased as a life or death issue - a modernize or be invaded and see the end of socialism problem. Thus, the real desired product of Soviet collectivization was not the quality of life for the peasants, but quantities of grain exported for foreign credit.
Sources:
Stalin, Joseph V. Dizzy with Success: Concerning Questions of the Collective-Farm Movement.
Tauger, Mark. Soviet Peasants and Collectivization, 1930-39: Resistance and Adaptation.
Cheremukhin, Anton, et al. The Industrialization and Economic Development of Russia through the Lens of a Neoclassical Growth Model.
Kuromiy, Hiraoki. Stalin's Industrial Revolution: Politics and Workers, 1928–1932.
last famine ever experienced despite centuries of frequent famines
I would assume this would be true for most if not all nations post-industrialization?