The internal logic that the founders relied on, and that produced what in retrospect looks a lot like elitism, was based in what historians refer to as "classical republicanism." It's a complex ideology that did not begin or end in early America, but I'll try to break it down in a simple way.

As the name implies, classical republican thinking looked to antiquity—especially the ancient republics—for examples of how republics function. The major lesson that classical republicans took from antiquity was that republics were inherently unstable and susceptible to corruption. They went through cycles of virtue and corruption, and the only thing that a republic could aim for was to stave off corruption for as long as possible. It was understood that the Roman Republic descended into corruption and decadence over time, leading to its demise. As classical republicans understood it, the only way to stave off corruption was to cultivate virtue.

In this schema, virtue had a very particular meaning. It didn't necessarily have a religious meaning, or even quite a moral one in the way we now think of it. Rather, virtue was the antithesis of both corruption and "fortune." Fortune, in turn, referred to chance, to the luck and risk of the marketplace. A merchant might be rich, but their wealth was due to fortune, rather than virtue. In contrast, the most "virtuous" people—and thus the ones most qualified to be leaders and voters—were ones who were free from dependence. Dependence might mean a lot of things. It might mean dependence on the marketplace (so even wealthy people without landed property were disqualified). It might mean dependence on a master (so slaves, servants, and apprentices were out) or a patriarch (leaving almost all women out). In this ideology, being dependent on others meant that you couldn't truly act for the benefit of the public good, because you needed to act out of self-interest. The most virtuous people, then, were men of independent means—especially those who owned property in land. But the matrix of virtue was not entirely exclusive to white men initially. In fact, in New Jersey single propertied women (independent in two senses) were allowed to vote for a brief period from 1797–1807. In a few New England states, some free blacks were allowed to vote for a time. To be sure, though, this was the exception rather than the rule.

Classical republicanism didn't last very long as an operating logic for the United States government. Its hierarchical arrangement of society roughly on the basis of status gave way over the course of the early nineteenth century to one based on biology that is more familiar to us today. Over time, race and gender became the primary means of excluding people from full citizenship. All white males became enfranchised, but that came at the expense of a system that was in some ways more fluid and empowering to non-white males.

Now, this is all fairly prescriptive, high-level thinking. In practice, individuals varied considerably in their approaches. But historians have shown that some version of this classical republican thinking pervaded the thinking of the founders in the American Revolution. It can go a long way to explaining both why the Founders operated in ways that seemed elitist, and why voters looked to wealthy men to be leaders.

Sources:

• Rosemarie Zagarri, Revolutionary Backlash: Women and Politics in the Early American Republic (Philadelphia: University of Pennsylvania Press, 2007). [on NJ women voting, and the switch from a status-based hierarchy to a biological one]

• J. G. A. Pocock, The Machiavellian Moment: Florentine Political Thought and the Atlantic Republican Tradition (Princeton: 1975). [one of the classic statements on classical republican thinking]

• Gordon S. Wood, The Radicalism of the American Revolution (New York: Vintage, 1991). [a book with some serious faults, but which does a good job of explaining the ideas behind classical republicanism]

• Drew R. McCoy, The Elusive Republic: Political Economy in Jeffersonian America (Chapel Hill: University of North Carolina Press, 1980). [this book does a great job of showing how Jefferson and Madison transformed classical republicanism to meet the realities of a modern world]