I think they were more concerned with the masses voting expressly FOR their own interest rather than for the greater societal good, more than anything. In Unruly Americans, Woody Holton argues that the voting public, larger after the Revolution than it would be at any time before the widespread adoption of universal white male suffrage, had been too free in voting themselves out of financial responsibility. Of particularly pressing concern was the distribution of the small about of cash in the fledgling US. As good societal patriarchs, many of the Framers and those of their social class had lent out money on interest, only to see intensely democratic state legislatures vote down interest rates or allow people who borrowed cash to repay with goods-in-kind. Of course, goods are not coin, and this whole system threatened to cripple the economy of the country.

Also remember that, as originally conceived, Americans could only vote for their Representatives in the lower house of Congress. Here, the democratic impulse of the time could be counterbalanced by the elite-dominated Senate. This was a system that viewed the electorate with suspicion, and moved to limit the damage done by a voting public that did not vote correctly.