Disclaimer: I am not an actual historian (a history student though) and answer mainly from memory, if that is against the rules of this sub, apologies.

Hitler's popularity in Germany was relatively big and the NSDAP reached over 40% in some of the free parliamentary elections of 1932/33. Most of this popularity came from economic depression that discredited the previous reigning parties as well as their response, which was mostly blaming each other and bickering over minor issues, such as a 1.5% increase in social security fees. The NSDAP pointed out these flaws and claimed that the problems of Germany came 1) from the inability and weakness of the established parties and 2) the Versailles treaty.

This is the first reason why there would be hesitation to intervene: Hitler's rise to popularity mainly came from his open opposition to Versailles, therefore intervening would only have proven his point of Germany being unfree and requiring a 'national awakening' to regain an equal status to France and the UK. Secondly, remember that Hitler originally came to power in a relatively democratic way. He was named Chancellor by President Hindenburg after having won over 40% of seats in the Parliament. This is obviously a mistake in hindsight but makes sense from his perspective - German governments were collapsing left and right, so naming the leader of the biggest party Chancellor made sense. The Ermächtigungsgesetz passed Parliament - obviously its content was undemocratic, but the process wasn't. Opposing intervention in a sovereign state for something its Parliament decided makes sense from the 1933 viewpoint. Thirdly, and I may be wrong here, but I think Versailles was criticised in the UK. The strictness of the Treaty, IIRC, was mainly driven by the French who wanted to see the Germans punished. The notion of "Erbfeindschaft" ("hereditary enemies") between Germany and France was somewhat present on both sides, and France fought two wars on own soil vs Germany in less than 50 years. They wanted to make Germany unable to fight a third one, a thought present in French thinking even after WW2.

Finally, there's an additional reason: in the same crisis, the KPD became much stronger as well. In 1923, the French already tried to force Germany into complying with Versailles and triggered a massive inflation crisis by doing so. There definitely was a fear that the same may happen again and, if Hitler was removed and the NSDAP discredited, the KPD would rise to power. For the UK (and to a lesser extent France), Hitler seemed like a lesser evil to the Communists, and removing him may have greatly helped the KPD.