Stalin was kept in power by the party. The source of his power was his control of the party apparatus. When most of his offers of resignation occurred, the 20s and 30s, Stalin was by and large not even well-known to much of the majority-rural population of the USSR.

The long and short of these offers is that no, they were not sincere on Stalin's part. Per Stephen Kotkin's narrative and characterization of Stalin, they are more equivalent to momentary temper tantrums from a proud yet chronically insecure man meant to reaffirm his ego in the most possibly public venue.

In extending his absolute control over the party apparatus, the real source of Stalin's power was his ability to make himself absolutely indispensable to his comrades within the party, on an organizational, ideological, and personal level. He had taken up many tedious behind-the-scenes organizational tasks during the civil war that predecessors had been unable to handle in order to build up his initial power base within the party. He mastered the language and thinking of Marxist ideology to portray himself as Lenin's most faithful pupil, and his course as the only orthodox course. He portrayed himself as the "radical centrist" of the party, the unbiased mediator who could successfully defuse dissent and disputes within the party and keep the USSR on a stable footing; he was the factionalist par excellence who somehow managed to brand himself as the premier anti-factionalist. He wove a vast web of patronage at all levels of the party - from his inner circle, to the ambitious ladder climbers in the middle, all the way down to the newest party members, thousands of people ultimately owed their social mobility and future career prospects to Stalin personally, many of these people coming from proletarian and peasant backgrounds who have had no comparable future under the old tsarist system. And Stalin himself and his closest allies always made sure to advertise their genuine proletarian credentials (even though, for Stalin himself, this was not strictly true, for Stalin had in truth been one of the intelligentsia, but his personal mannerisms were certainly proletarian) in sharp contrast to the likes of the arrogant, aloof, condescending disdain of Trotsky and the self-absorbed aggrandizement of Zinoviev. Stalin's regular performative shows of humility were refreshing by comparison.

The party did, in fact, have the power to remove Stalin. His offers of resignation were never sincere, but the power of the party to remove him by vote existed. It was a constant sword of Damocles hanging over Stalin's head - the call by Lenin's "testament" (very likely a fabrication by Krupskaya, Kotkin does considerable legwork to show that its supposed transcription by Lenin's secretariat does not fit very easily into the timeline of his incapacitation) to "remove" him as General Secretary dogged him constantly from its first appearance. But, nobody wanted to remove him, not even his rivals such as Zinoviev, Kamenev, or Rykov. He was too valuable an organizer, theorist, and leader to remove.

Of course, the existence of Stalin's personal dictatorship meant that any real alternative to his preferred course - as opposed to a mere intellectual exercise - had to trump his power, either by outvoting him, because members of his faction defected, or by removing him. Bukharin had tried such a maneuver and failed, but when Stalin, by offering to resign, handed Rykov the opportunity, he failed to seize it. Perhaps Rykov acted out of political self-preservation, given Stalin's power and vengeful disposition. But Rykov and others in the politburo had come to see not only a prickly, self-centered, often morose, vindictive person in Stalin, but also an indomitable Communist and leader of inner strength, utterly dedicated to Lenin's ideas, able to carry the entire apparatus, the country, and the cause of the world revolution on his back. Stalin displayed a strategic mind, which had its cruelties - sizing up the weaknesses of Bukharin for sadistic as well as political purposes - but also its payoffs for managing the nationalities and regional party machines. Additionally, the group arrayed around Stalin was incomparably below him. Orjonikidze was no strategist, and in constant poor health; Voroshilov was no military man, and he knew it; Kirov had a public politician's touch but was given to laziness and womanizing; Kaganovich was an organizer of talent but barely educated; Mikoyan worshiped Stalin, not just for careerist reasons, but because he was young, Kalinin was underestimated, but also no Stalin; Molotov could flex some political muscles, but even he operated in Stalin's shadow. Stalin's dark side had become no small matter to manage, but managing entirely without his leadership?

-Stalin: Paradoxes of Power (1878-1928), by Stephen Kotkin