I remember reading a quote by Nikolai Yezhov (head of NKVD) in which he said:
"Better that ten innocent people should suffer than one spy get away. When you chop wood, chips fly"
Which raised the question to me - did they even catch any spies?
Witch hunts, inquisition, purges have their own dynamic. Acting on a belief that there are witches, heretics, spies and traitors some are found, made to confess, and punished. Confessions usually include naming of accomplices, those getting the same treatment and so on. One in that process, those running it quickly end up in a dilemma, continue even when it seems to get out of hand or realize that the initial threat was insignificant or non-existent and that so many people were already punished for nothing. Second option is very difficult to admit, personally and publicly.
In Stalinist purges there is that aspect, there is a factor of Stalins' paranoia, there is an element of power struggle. Those are all significant, but seem to me insufficient as explanations. I consider purges to largely be a popular movement.
Stalin's purges begun after a period of New Economic Policy, that seemed to reverse much of revolutionary gains and reintroduce both inequality and exploitation. Also, revolutionary generation of communists was entrenched in mid and high management positions, and that blocked vertical mobility for newly educated generations. Therefore, purges were not only largely supported, but were in many aspects a popular movement. Stalin dealt with whomever he paranoidly thought of as potential opposition, a degree of revolutionary egalitarianism was restored, and vertical mobility once again became possible. I believe that it was the purges that largely provided a strong enough support for Stalin to weather the Second world war. All those who were promoted through purges, felt their social position tied to the figure of Stalin.
(I don't have one single book or an author to support this view of Stalinism and purges, at least not so explicitly. But it kind of comes through works like Fitzpatrick's "Russian Revolution", and especially "Education and Social Mobility in Soviet Union 1921-1934", Neumann's "The Communist Youth League and the Transformation of the Soviet Union 1917-1932", and some others, although I haven't read anyone explicitly calling the Great purge a popular movement.)