Not the Civil War per se, but that great original sin of slavery was prominent as a deciding factor for the land being returned to Virginia in 1846 in a process known as Retrocession.

But yes to state outright, when originally selected the new capitol would be a roughly 10x10mi square with land on both sides of the Potomac. And including the then growing river towns of Alexandria VA, and Georgetown MD. The spot is close to the falls line on the Potomac, meaning they are the furthest ports up river one could get by boat before needing to build canals and such or it just gets too difficult. Many cities along the east coast are located in similar manner, Richmond is another example.

In 1801 the DC Organic act was passed by Congress once the government took up residence, it did a number of things including ending the transitional voting period when residents could still vote in their old states and districts and lost the ability to vote for full representation in Congress completely and for the President until the 23rd Amendment was passed in 1961. It also organized the District into the city of Washington, the towns of Georgetown and Alexandria, and the rest into the Counties of Washington on the MD side and Alexandria on the VA side.

But there was very little equality on the separate sides of the river in one key regard. While Alexandria was a growing port for shipping, and slave auctions, it was losing steam compared to Georgetown. Helped along by further legislation that prevented the investment in and construction of public buildings and federal offices such as they existed, anywhere but on the MD side of the river.

Especially as Gtown began to benefit even more as the transport infrastructure improved. George Washington had been a main supporter of early canal efforts, including the preserved Patowmach Canal which got around Great Falls and gave access all the way too the Shenandoah in the late 1780's. This link would be much expanded on by the Chesapeake and Ohio Canal that was mostly built on the MD side beginning in 1828 with the terminus being in Cumberland MD. You also in 1815 had the Washington City Canal which ran from Buzzard Point/where the War College is now, up in front of the Capitol, down what is now Constitution Ave in front of the White House, and let out closer to Georgetown. Avoiding an area of the Potomac that was particularly prone to sand bars, and connecting the docks of Gtown to where the majority of the new cities docks were located. The Alexandria Canal would only open in 1843, it would ran from Old Town, along the Potomac past the modern sites of Crystal City(and soon Amazon HQ2), the Pentagon, Arlington Cemetery, and near modern Rosslyn crossed the Potomac via bridge where it tied into the C&O Canal. But this was in many ways too little too late.

And while frustrating it was still not the final straw. That would be the prospect of increasing threats to the South's peculiar institution of slavery. As abolitionist sentiment rose in the nation, though concentrated in the Mid Atlantic, New England, and Old Northwest, the residents of Alexandria DC could see their future. Should Yankee congressmen push through a ban on slave auctions, let alone slavery entirely in the capital then that would be it for their keeping up. Meanwhile Virginia would be happy to have a profitable town back, and those voters and tax payers as well.

So sentiments finally came to a head in 1846, though groups had agitated for a return to VA almost since the federal govt took up residence! Virginia passed a bill stating they would take Alexandria back if Congress allowed it, Congress passed a bill to send to President Polk in July, which he signed allowing for a referendum by the residents on the matter which passed with near 75% of the vote. And so in September 1846 the portion of the District South of the Potomac was returned to Virginia.

It is notable that while Slavery was not yet the all consuming matter it would become over the next decade, it was also already occupying a major part of the debate and decision making, though we cannot ignore simple discontent with what Alexandrians felt was a raw deal of the combo of neglect of public works/investment on top of disenfranchisement.

Some additional easy reading on the topic:

https://historyengine.richmond.edu/episodes/view/665

https://blogs.weta.org/boundarystones/2016/07/08/alexandria-retrocession-1846

And on canals of the Potomac: https://www.alexandriava.gov/uploadedFiles/historic/info/history/HistoryOfTheAlexandriaCanalBrochure.pdf