I'll be honest, I had some trouble trying to formulate an answer to this question at first. The thing is, the answer can be as simple as "because they wanted an accessible, proletarian art style," but it can also be as complex as a book. I'm going to attack the question from the angle of Socialist Realism's origins and development, because that's the part I know best, and also context always helps. But the answer will depend on who's answering, and what they're knowledgeable about.

What Socialist Realism Was and What It Wasn't

That said, first, what was (is?) Socialist Realism?

I don't want to repeat stuff you know back at you, but it is a little bit more complex than just realistic portrayals of life under socialism. To a skeptic, or a critic, it may not even seem very realistic at all — you may have seen Evdokiya Usikova's Lenin With Villagers and thought it was sentimental, corny fluff, or seen this mural from the Kulturpalast in Dresden and thought it was far too cluttered and stylized to be considered realist or naturalist art. But the point of Socialist Realism is not necessarily to meet our definition of objectivity, in terms of style or subject matter.

Rather, the goal of Socialist Realism is to portray "reality in its revolutionary development." That quote comes from Andrei Zhdanov's speech at the 1934 Soviet Writers' Congress (which you can find here), where he laid out what the goal of Soviet art should be. In his telling, "objective reality," for a socialist artist, was a bit of a red herring. Instead, what should matter to the artist is what would ultimately matter to a citizen of a future communist utopia looking back on the past — hence "revolutionary development." It was to show the people of the present what their everyday, mundane lives meant in the broader arc of Marxist history, what their labor would produce, and thereby inspire them to work harder, to develop themselves, to become the people they saw in Socialist Realist art.

Stalin famously said that "writers [were] the engineers of human souls", meaning that they were responsible for creating the new Soviet person who would populate their utopia. For someone trying to engineer the human soul, then, art should help bring that utopian future into being by providing a straightforward model of it that everybody could understand and work to achieve. So you can think of it as bringing reality and utopia together, first by making utopia more accessible to the people of reality, and in doing so, by making reality more like utopia.

That influenced the choice of proper subject matter: people involved in socialist labor building the future, workers and peasants developing political consciousness, guiding revolutionary figures in thought. So that subject matter is already a limiting factor, and it requires a certain degree of conformity to realistic style. But it doesn't rule out more impressionist portrayals of revolutionary development, or even metaphorical, abstract portrayals. And that finally brings us to the meat of your question.

The Meat of Your Question

Why couldn't Socialist Realist subject matter be portrayed more abstractly? Why couldn't the progress of history be shown in metaphor? Why couldn't the revolution be pushed forward some other way?

The answer is, it had already been tried, and Zhdanov, Stalin, and the other members of the Stalin faction didn't like it.

The second revolution of 1917, in Soviet historiography, divided history into "before" and "after." That's directly relevant to Socialist Realism, of course; it meant that Soviet citizens were now on the path to the end of history, and that Soviet artists had the responsibility to show and encourage that through Socialist Realism. But Socialist Realism wasn't codified until 1932, and depending on how you define it, it didn't even exist until the mid-to-late '20s. So it also meant that, before Socialist Realism, Soviet artists defined themselves by their radicalism, by their boundary-pushing nature, by being at the forefront of history — the avant-garde.

The Avant-Garde and Why Stalin Repressed It

Before Socialist Realism emerged, several artistic genres and styles floated around in the petri dish of post-revolutionary culture: collectively, the Soviet avant-garde. The artists of the avant-garde expressed the radical nature of the society they wanted to build by being equally radical stylistically. I would describe the art of the period as being characterized by an attempt to break free of the constraints of representation, in the same way that the revolution had thrown off the shackles of capitalism, and move towards an art of pure form and color. That, put simply, means they used simple geometric shapes, forms that suggested motion and dynamism, and vibrant colors. For example, there was Suprematist art like El Lissitsky's Beat the Whites with the Red Wedge and Malevich's Black Trapezium and Red Square, or Constructivist architecture like Moisei Ginzburg's Narkomfin building and Ilya Golosov's Zuev Workers' Club.

Those were, in the 1920s, at least, acceptable ways to make socialist art while using abstraction. What changed? The cultural and political radicalism of the 1920s became less acceptable as Stalin tightened his grip on power. Partly, Stalin and his supporters thought that avant-garde art was pretentious and irrelevant to the average worker. It was something for the intelligentsia to debate, not for workers to appreciate and derive hope from. But it was also that, as radical, utopian policy proposals were forced out of the public sphere, radical art went with it. Art and politics were a continuum. It wasn't far from art to architecture to urban planning to internal policy. The Moscow Metro is an example: it began with a decree authorizing the redesign of Moscow as a blow against Anti-urbanist political radicalism, and over the course of construction and of its artistic execution, it became a showcase of Socialist Realist art as a rejection of avant-gardism.

And the Metro brings me to what happened in the 1930s. The avant-garde experienced a heyday in roughly 1928–1931, but a quick decline after that. In April 1932, the several avant-garde literary and visual arts organizations of the 1920s, such as RAPP, Proletkult, and AKhRR, were disbanded and replaced by "creative unions" like the Union of Soviet Writers, the Union of Soviet Artists, and the Union of Soviet Architects. The consolidation was supposed to put all artists under the same democratic representative structure, the union. But the leaders of each union, picked by the Soviet leadership, were to nudge members towards conformity with Socialist Realism, which was also made official in April 1932.

So at least in the 1930s, there was no longer such a thing as a pro-Soviet painting that relied too heavily on abstraction or surrealism. By creating art that the average worker could not understand and that would not help them understand their role in the march of history, it was automatically bourgeois in character.

Conclusion

This was an extreme, of course. Under Khrushchev, there was more room to experiment, and even after Brezhnev clamped down on political discourse in the 1960s, more and more non-representational and avant-garde art emerged — arguably even more under Brezhnev than Khrushchev. But I don't know as much about that, and certainly don't know anything about the state of the historiography, so I ought to leave it there.

To sum it all up, if you're pressed for time: subject matter was not the only part of proper socialist art, according to Socialist Realism. Socialist art needed to be politically helpful and contribute to the victory of communism, by improving workers' consciousness of their role in history. If they couldn't easily understand it or if it was too strange and new, it couldn't do that. So Soviet art had to be mostly representational and realistic just as much as it had to be "pro-Soviet", because the two were philosophically linked.