Aside from Minnesota, where the river begins and therefore can't really be a border, Louisiana is the only state that the river cuts through. Why is this?
As of the Louisiana purchase the Eastern borders of the territory were ill defined and both France and Spain had competing claims on the area you recognized.
In 1812 the US Annexed that region, the Mobile District, that looks like a foot at the bottom of Mississippi. Before that it was contested Spanish territory.
The border of Louisiana and Miss. in that area is the Pearl River which forms the border until you reach the 31st parallel.
citations:
Schoultz, Lars (1998). Beneath the United States
Haynes, Robert V. (2010). The Mississippi Territory and the Southwest Frontier