The key differences were on issues of nationalism and localism. The best way to compare the two parties is to compare their ideological leaders. Alexander Hamilton was a driving force behind the Federalists while Thomas Jefferson was the founder of the old Republican Party (Democratic-Republican, Jeffersonian Republican).

Alexander Hamilton served as Washington's first Secretary of the Treasury (he is on the $10 and the treasury building is on the back of that bill) and he put forward a number of centralizing plans for economics, including a tariff, a national bank, and bounties for industrial production. He wanted to use the power of the newly minted federal government to develop the economic and industrial power of the United States along the lines of the British.

Jefferson (First Secretary of State under Washington), on the other hand, was far more in favor of local economics. He supported the yeoman farmers. He wanted America to be run by the states more than the federal government, no tariff as he and his fellows were exporters of raw materials and Europe had more demand for Tobacco and Cotton (a new industry for the USA thanks to the Cotton Gin), and other cash crops than the USA had. He was also more in line with the French over the British.

From about 1796 through about 1815 the two parties were dramatically opposed the key issues of economics (plantations or factories), location of political power (national vs local), banking (yes or no), and trade policy (protect our factories or export our goods).

By the war of 1812 the Federalist party was in shambles. No presidential victory since 1796, only Marshall on the supreme court to have any power, an aborted attempt to break New England off as its own country failed and the party dissolved. The Republican party was unopposed in the election of 1820.

Some good sources to read on these differences would be found by looking up Hamilton's economic plans, the election of 1800 which was a major battle between the two parties for control of the 15 states and the federal government, and the even the war of 1812 which we see the beginning of the end of the Federalist Party.

Text sources archive: http://www.digitalhistory.uh.edu/era.cfm?eraID=4&smtID=3

Hamilton's Manufacturing plan: (federalist economic policy) http://www.digitalhistory.uh.edu/disp_textbook.cfm?smtID=3&psid=265

Jefferson and Madison's Kentucky resolution: (a reaction to federalist legal policies) http://www.digitalhistory.uh.edu/disp_textbook.cfm?smtID=3&psid=4127

I hope these help answer your question.