I'm no expert so take what I say with a grain of salt.

From what I understand, Richmond was chosen because Virginia was one of the later states to secede from the Union, waiting until after Fort Sumpter, and offering them the capital was a way to induce them. Plus, Richmond was in one of the few, heavily industrialized areas of the south, and was particularly near many railroads AND the James River, which was great for the Confederate brass, most of whom were from Virginia, or were nearby if they were part of the US Army before the war. And, Virginia, especially Northern Virginia, was heavily populated, and would serve to create a big army near the front lines quickly. Remember too, most of the Confederate leaders were formerly US politicians too, and had gotten used to the climate and city atmosphere of Washington, both of which Richmond had, and the "backwoods" town of Montgomery didn't. But primarily I think it came down to the fact that the area around Richmond had better industrial production output as opposed to Montgomery, as well as Montgomery's relative lack of railroad and other transportation options compared to Richmond.

To talk about New Orleans, honestly, most of the city was opposed to the war. There wasn't a whole lot of support in the city for secession, since it got most of its wealth from upriver trade with regions that were not seceding - in fact, most of the Mississippi River valley was opposed to secession. Now, this isn't to say they were for abolition - absolutely not, they were entirely in favor of slavery, but they did not want to lose their lucrative trade from the northern region. New Orleans (the city itself) was taken by the Union without firing a single shot, although there was a naval battle at the mouth of the Mississippi. Lee Sandlin's "Wicked River" is an excellent book and has an entire chapter dedicated to the Civil War along the Mississippi, though it focuses on the major siege at Vicksburg. In short, New Orleans being taken so quickly and easily shows how much the populace was opposed to the war as well as how vulnerable it was (even more vulnerable than Richmond, even with its proximity to the Union capital.)

Charleston was much like New Orleans, in that it was quite vulnerable as a port city. You have to remember that outside the blockade runners and the CSS Virginia that you always hear about, the CSA didn't have much of a navy, and wouldn't be able to defend a port city.

So, when it comes down to it, Richmond was just the best option overall. Virginia was the most powerful state in the Confederacy, Richmond was the third largest city in the CSA, it was relatively industrialized compared to the rest of the South, and it was more easily defensible than a coastal city, as the Confederate Army could better defend a city than the Confederate Navy. Plus, remember that it wasn't just the Confederates that were so close to the enemy. Washington, D.C. lie literally just across the border from hostile territory.