Why was the American civil war fought mainly with muzzle-loading muskets when far more developed breech-loading weapons like the Prussian needle gun and the kammerlader had been around for decades?

I posted here about the same question regarding why the US Army kept the 1903 Springfield around. Much of that answer retroactively applies to the US Army on the eve of the Civil War.

The US Army was tiny and conservative, and the muzzle-loading rifles used by many of its men perfectly served its function before the war and even during.

Despite what may seem like a no-brainer change (more bullets downrange is better, right?), there was a fierce debate among high ranking men in the army about whether the advantages of a greater firing rate were actually better than a slower, more methodical approach. I'll sum up the two camps a bit:

Traditionalists, we'll call them, believed that a high rate of fire was wasteful, and only encouraged men to burn through their ammunition without effect - e.g. they wouldn't hit or kill more enemies even if the rate of fire was higher. Breech loaders were also far more expensive to produce, much more difficult to maintain, and the open breech allowed muck, mud, dirt, and other obstructions to enter the breech mechanism and the barrel far more easily than a closed breech. All of this, according to the traditionalists, would lead to an army that was less effective despite the ideally increased rate of fire.

There is actually quite a lot that backed up this mode of thought. One of my favorites to reference is the Battle of the Rosebud which occurred in 1876, just months before the Battle of The Little Bighorn (it concerns the use of single-shot breech loaders, but the doctrine is the same) The US Army forces fired more than 50,000 shots at the enemy, and only reported 100 enemy casualties. Solders got excited. War is hard. They fired blindly only to be comforted by firing, and the job of the officers all the way up the chain was to limit terrified soldiers' ability to damage the army. So stick with a slower rifle, keep the soldiers concentrating on making their shots effective.

Innovators, however, argued that an increased rate of fire gave at least as many advantages as disadvantages, and those disadvantages could be overcome in training. A higher rate of fire didn't necessarily need to lead to more casualties, but to a more aggressive and kinetic battlefield force. More rounds downrange meant a more uncomfortable enemy force, it meant taking and keeping attacking initiative, it meant shutting down the possibility of charges and more opportunities for flanking movements. That said, Innovators were still very conservative, compared to modern military doctrine, but their point of view was, slowly, adopted with many caveats.

You could more or less project this argument forward about a hundred years and people in the US Army were still arguing for versions of both of these opinions.

It should also be pointed out that the fact that breech loaders existed did not mean that they were inherently superior weapons, or that their tactical options were any more battlefield-effective than those allowed by muzzle-loaders. The Needle was used for a long time before the Austro-Prussian War, which demonstrated to many that the rapid rate of fire afforded by the breech-loading Needle was far superior to the Austrian's muzzle-loader.

Half a decade later, though, saw the Needle up against the French Chassepot, which was by most accounts a far superior rifle in range, stopping power and reliability. This is to demonstrate that battlefield effectiveness is a multifaceted question, which relies far more on doctrine, training, leader etc than it does on individual rifles.

The US Army, ultimately, stayed relatively conservative for a long time based heavily on the fact that it was a very small service, and funding for that service, unlike now, was a politically controversial question. Old Whig ideas regarding standing armies stayed around, in one form or another, long past the time "Whiggery" was remembered as a viable political ideal.

David F. Butler, United States Firearms: The First Century: 1776-1875 provides a great deal of information regarding the process of selecting, testing, and providing firearms, along with a number of interesting discussions of the doctrinal approach of the US Army.