I assume there was some logic to this, but the most I can find is that as colonies both claimed the entire river, and then Virginia relinquished its claim, but no explanation for why they were willing to.
This arrangement is pretty common along river boundaries. There are many practical reasons to place the border along one side or the other, among them licensing of boats and fishing, and that it is harder to find the middle than to find the riverbank.
In reality, though, it comes down to what the charter/court/legislature says.
Sometimes there is political pressure and sometimes deals are struck in setting the boundary. For example there was a 17th century border related conflict between Baltimore and Penn in trying to decide what water access Pennsylvania should get.
In the case of states admitted to the union, an already admitted state has much more political power, and can trade its support for territory. This is why Ohio got Toledo. Michigan was in turn allowed to grab the U.P. from Wisconsin.
I can't answer to Potomac River, but the boundary between the distant parts Virginia that are now Kentucky and what was to become the state Ohio was readjudicated in 1981 so that Kentucky gets the river, but only up to the 1792 waterline on the Ohio side. This is tricky because the river has changed course and in many places this is now in the middle of the river.
In 1910, part of the boundary along the West Virginia and Maryland was decided to be the south bank in Maryland v West Virginia.