The history of the electoral college is fairly interesting. The Constitution actually makes no mention of how states conduct their elections, save for Senators being elected "by the people of the state" (U.S. Const. Amend. XVII), and that each state must have a "republican" form of government (U.S. Const. Article IV Section 4). The electoral college is essentially a gathering of "ambassadors" representing the various states, much like Congress. In fact, the wording used in Article 2, Section 1, Clause 2, refers to states "appointing" an elector in a manner prescribed by the individual legislatures. A state could, in theory, just appoint some state officials and tell them to vote however they like. Now, granted, the whole "election by district" concept was around back then, even being proposed by the New York Legislature as a constitutional amendment in 1802, so this is somewhat irrelevant. The point I'm trying to make is that our current system is very far removed from the original concept of an electoral college, and that the entire issue of faithless and faithful electors doesn't necessarily have to exist. In fact, the first presidential election in 1788 saw many state legislatures vote amongst themselves to select electors. The New York legislature deadlocked, and only 6 of the other 10 states that actually voted used a popular vote system.

With that out of the way, let's see what some founding fathers had to say on the matter.

Alexander Hamilton, in Federalist No. 68, wrote:

I venture somewhat further, and hesitate not to affirm, that if the manner of it be not perfect, it is at least excellent. It unites in an eminent degree all the advantages, the union of which was to be wished for.

It was desirable that the sense of the people should operate in the choice of the person to whom so important a trust was to be confided. ...

It was equally desirable, that the immediate election should be made by men most capable of analyzing the qualities adapted to the station, and acting under circumstances favorable to deliberation, and to a judicious combination of all the reasons and inducements which were proper to govern their choice. A small number of persons, selected by their fellow-citizens from the general mass, will be most likely to possess the information and discernment requisite to such complicated investigations.

It's clear that Hamilton supported the direct interpretation of the electoral system: That of a group of "ambassadors" from each state, gathering to examine the candidates and choose themselves which is most qualified to be president.

With that done with, let's take a look back at Madison and his "elector per district" idea. In a letter to George Hay, he wrote of the proposed amendment and the "general ticket" system, in which the people of a state would choose a slate of electors in a state-wide election based on who the electors pledged to vote for (this is what we use today).

I agree entirely with you in thinking that the election of Presidential Electors by districts, is an amendment very proper to be brought forward at the same time with that relating to the eventual choice of President by the H. of Reps. The district mode was mostly, if not exclusively in view when the Constitution was framed & adopted; and was exchanged for the general ticket & the Legislative election, as the only expedient for baffling the policy of the particular States which had set the example. A constitutional establishment of that mode will doubtless aid in reconciling the smaller States to the other change which they will regard as a concession on their part. And it may not be without a value in another important respect. The States when voting for President by general tickets or by their Legislatures, are a string of beeds: When they make their elections by districts, some of these differing in sentiment from others, and sympathizing with that of districts in other States, they are so knit together as to break the force of those Geographical & other noxious parties which might render the repulsive too strong for the cohesive tendencies within the political System.

While Hamilton and Madison certainly don't represent the attitude of every founding father, it seems clear that the current system was not what a significant portion of the delegates intended when the Constitution was drafted. Instead, the intent seemed to be for a greater emphasis on the choices of an individual elector (who would be elected ideally as a trusted member of their district), and for a more direct influence of individual districts on the vote (rather than being washed out by the rest of the state). To directly address your question, the issue of faithless electors simply did not exist at the time, because the system in which every elector pledges to a candidate and is elected by the state as a whole was conceived and implemented after the Constitution was first adopted.

Edit: typos that I just noticed