There were four central figures around Hitler who held power in several different ways.

Arguably the most powerful was Himmler who was in charge of the SS. Göring however was appointed Hitler's second in-command and his successor. Bormann was Hitler's personal sectary and it was impossible to have a meeting with Hitler (post-1944) unless it had been sanctioned by Bormann. The only person who didn't need to go to Hitler via Bormann was Speer who, as the Reich declined, took over many of the critical facets of maintaining the war effort and held the falling state together domestically as it began to collapse.

Arguably in terms of political power and ability to exercise it the most powerful was Himmer as he ran the SS of which the Gestapo were subservient.