His rise has never been that clear to me. Why did he come to power after Lenin? What happened to Trotsky? It seemed like Trotsky was in a good position to succeed Lenin but then Stalin ended up on top. I know that Stalin helped Lenin raise money for the Bolsheviks but their relationship deteriorated.
Stalin was able to gain power by maneuvering among factions within the Communist Party while simultaneously, via his office as General Secretary of the party, accruing people in party leadership positions with personal loyalty to him — as well as just being tremendously patient. It was a complicated process and, as you note, Lenin’s relationship with Stalin had deteriorated shortly before Lenin’s death. I can give the broad strokes, but it’s even more complicated than I’ll lay out here.
Factions had emerged in the party at the end of the civil war primarily over the New Economic Policy (NEP), a concession made by Lenin to allow for a limited return of capitalism to rebuild markets, particularly in agriculture, that had been destroyed during the war due to harsh requisition policies (so-called war communism). Trotsky led the Left Opposition of party members opposed to the NEP. Upon Lenin’s death, Stalin, who was already General Secretary, joined Lev Kamenev, then a deputy to Lenin, and Grigori Zinoviev, chair of the Comintern, in a troika, or triumvirate, to lead the USSR. Together, the three succeeded in marginalizing Trotsky, although he remained in a powerful position.
Over the subsequent year or two, Stalin drifted toward the Right Opposition, led by Alexey Rykov, successor to Lenin as head of the Council of People’s Commissars, Mikhail Tomsky, head of the trade unions, and Nikolai Bukharin, editor of Pravda. The Right Opposition was a party faction that strongly favored continuation of the NEP, which dovetailed well at the time with Stalin’s enunciated goal of Socialism in One Country, compared to Trotsky’s internationalist theory of Permanent Revolution.
With the Right Opposition, Stalin managed not only to dissolve the ruling troika and push Kamenev and Zinoviev from power but also to, eventually, expel both men, as well as Trotsky, from the party, by 1927. While Zinoviev and Kamenev were merely expelled, Trotsky was exiled, first internally and then to Norway. He eventually settled in Mexico, where he was assassinated in Stalin’s orders in 1940.
With Trotsky, Kamenev, and Zinoviev neutralized, Stalin then turned on the Right Opposition. In this case, Stalin now came out against the NEP, arguing that the country needed to return to socialism. Having built a strong base among the party leadership and cadres, Stalin successfully expelled the Right Opposition members by 1929, by which point he had already begun the first Five Year Plan to rapidly industrialize the country and bring agriculture back under government control.
Although Stalin’s power was unprecedented in 1929, it was really with the purges of the party in the mid to late 1930s, during which all of the aforementioned former party members were arrested, tried, and executed, that he reached the apex of state power, mainly by demonstrating the extent to which he was willing to use violence within the party.
There are many good biographies of Stalin that cover these years in great detail. Stephen Kotkin is two volumes into a planned three-volume bio, the second volume of which covers the above period. A shorter, more concise version can be found in Oleg Khlevniuk’s single volume biography.