As with most things involving Washington D.C., the decision of where to found the Federal city was rife with politics. The downvoted points below do have some truth to them, but it's not the entire story.

The idea of a designated district for Congress had existed for years. Benjamin Rush wrote to Colonel Morgan in Nov. 1779 that "Congress have resolved to leave Philadelphia on the 1st day of next May. Some of the members talk of purchasing a few square miles of territory near princeton and erecting public offices and buildings of all kinds for their accommodation upon it, a more central - healthy - & plentiful spot I believe cannot be found on the continent."

Many towns vied to host the Capital, including Georgetown, Princeton, Williamsburg, and approximately 19 others. The final two contestants were sites near the Potomac and the Delaware rivers.^1 These choices were split along Northern and Southern preferences, each would benefit one side far more than the other.

At the same time, a parallel debate was occurring in Congress involving Revolutionary War debt. A deal was brokered between Hamilton and Madison in which the Northern delegates would agree to the Potomac site if the Southern delegates would agree to what would later be termed the Assumption Act(s).

To quote Jefferson directly:

I proposed to him (Hamilton) however to dine with me the next day, and I would invite another friend or two, bring them into conference together, and I thought it impossible that reasonable men, consulting together coolly, could fail, by some mutual sacrifices of opinion, to form a compromise which was to save the union. The discussion took place. I could take no part in it, but an exhortatory one, because I was a stranger to the circumstances which should govern it. But it was finally agreed that, whatever importance had been attached to the rejection of this proposition, the preservation of the union, & of concord among the states was more important, and that therefore it would be better that the vote of rejection should be rescinded, to effect which some members should change their votes. But it was observed that this pill would be peculiarly bitter to the Southern States, and that some concomitant measure should be adopted to sweeten it a little to them. There had before been propositions to fix the seat of government either at Philadelphia, or at Georgetown on the Potomac; and it was thought that by giving it to Philadelphia for ten years, and to Georgetown permanently afterwards, this might, as an anodyne, calm in some degree the ferment which might be excited by the other measure alone. So two of the Potomac members (White & Lee, but White with a revulsion of stomach almost convulsive) agreed to change their votes, & Hamilton undertook to carry the other point.

In the end, Georgetown was selected. Health issues notwithstanding, the Potomac made it an excellent site for a large city and a border was soon carved out of the surrounding states.

^1 It should be noted that the House favored a site on the nearby Susquehanna River while the Senate favored the Delaware. This disagreement was resolved by the selection of the Potomac site.

Harbaugh, M., & Harbaugh, D. (2013). The first forty years of Washington DC architecture. Houston, TX: Danzmark Productions.