I would argue that the two party system became institutionalized during the Jacksonian period of 1820s to 1840s. This period saw the rise of the second party system after the collapse of the Federalist party during the "Era of Good Feelings" and Democratic-Republican governments. This period had several different forces come together than legitimized political parties and created a precedent of the two-party system.

One factor that contributed was the rise in political participation. More than 3 times as many people voted in the presidential election of 1828 as did in 1824 due to expanded voting rights. In most states voting rights expanded from property owners (or tax payers) to all adult white males.

At the same time, America was experiencing economic changes including the transportation revolution (things like the Erie Canal) which allowed political parties to spread their message to a larger number of people.

The candidates elected during this period used the "spoils system" which rewarded political office to people who helped their candidate get elected. This encouraged people to not only get involved in politics but to pick a side and remain fiercely loyal to it.

Entering the Jacksonian period there was only one political party, the Democratic-Republicans. Andrew Jackson's administration would essentially fracture this single national unit in two: those that supported him and his policies, and those that opposed them. The group that supported Jackson became the Democrats, and all of the groups that opposed Jackson formed a sort of coalition government called the Whig Party. Their name came from the British Whig party that opposed monarchy (the Americans used this name to compare Jackson to a King).

Because this period was the emergence of a democratic citizenship, it created a legitimized precedent of two political parties. There was also a two party system prior to this period (the Federalists and Democratic-Republicans) but neither side fully recognized political parties as legitimate and positive. The Jacksonian period legitimized political parties and created a precedent of having two major parties that disagreed on several issues. It also legitimized the loyal opposition that was critical of the government in power.

America emerged from the Jacksonian period politically active and even had several elections where a third party played a significant role as slavery became a hot button issue. Because the two major political parties tried to avoid slavery as they both desired national support, the issue became so great that a third party addressed it (first Liberty, and then Free-Soil party) which eventually crashed the two party system of the Jacksonian Period.

I have already had to remove a few poor quality or speculative answers from this thread, all of which have been based around a fundamental misunderstanding of the question. For clarity, /u/5ubbak's question specifies:

Even in countries where only two parties have a decent shot at controlling the government, you expect some other parties to make up a significant fraction of the legislative body, compete in all the elections, and probably hold local executive offices

This question does not ask why only two political parties have historically been able to form a government; it is asking why the US has a two-party system so entrenched that only two parties have any significant kind of elected representation. Those are two very different questions, and the latter cannot be explained simply by single-winner plurality (as all answers thus far have suggested), which comparative psephology shows us does not automatically create such total domination.

A good answer can certainly discuss the electoral system, but should also consider the specific historical context that can offer us insight into why two-party domination is so strong. If you cannot deal substantially with that question, or if you answer boils down to a simple explanation of first past the post, then it is not sufficient for our purposes here.