Most of this will pertain to the American Civil War, as that's what I'm most familiar with.

Ammunition was a concern. The standard US army pattern cartridge box carried forty rounds of ammunition. Sometimes soldiers would carry more in their pockets or haversacks, when it was available, and of course a civil-war era muzzle-loader couldn't burn through ammunition at the same rate as a modern assault rifle, or even a semi-automatic or bolt-action rifle of the world wars. At three rounds a minute, the standard rate of fire quoted for well-drilled troops, the forty rounds in the box are enough for about 33 minutes of sustained firing--although they'd rarely fire at full speed for that entire time.

There are accounts of units running low on ammo, or out entirely. At Second Manassas (Bull Run) a Louisiana regiment defending a rail cut resorted to throwing rocks down at Union attackers when the ammunition was running out. There is another instance which I believe occurred during Jackson's campaign in the Shenandoah Valley in 1862, though I haven't been able to track down where exactly right now (most of my books aren't available to me at the moment), where the officers walked the soldiers through the step-by-step loading procedure in the drill manual--meant for training rather than the battlefield--in order to deliberately slow down the soldiers' rate of fire and conserve their dwindling ammunition supply. Low ammunition was a factor in Joshua Chamberlain's decision to fix bayonets and counterattack with the 20th Maine on Little Round Top at Gettysburg. Of course, these sorts of instances which get written down and noted are probably less common than the situations where a unit that was out of ammunition fell back and resupplied--it was the cases where that was impossible due to the situation (such as the tenuous status of the Union left at Gettysburg) where these sort of desperate measures took place. Also, one thing to keep in mind, in this era a unit's combat power--its ability to maneuver, deliver effective fire, and respond to orders--depended on keeping its formation and organization. By the time a unit had been in combat long enough to run out of ammo, chances are it was disordered enough to need to be pulled off the line and rested/reorganized anyway. This is one reason attacks tended to take place with several brigades one after the other, instead of all at once--a unit's time in combat was inherently limited, and without a supporting formation to relieve it any gains an attack made tended to be temporary.

I don't know how this sub feels about reenactors' personal experience, but I feel it could be helpful here. Obvious caveats that a reenactment is not a mirror of the conditions of an actual battle. I am a Civil War reenactor, and if I go onto the field with a full cartridge box, I have never actually gotten close to running out of ammunition. In fact, even during a battle that feels very "hot" with quite a bit of heavy firing, you actually use a surprisingly small amount of ammunition--sometimes fewer than 10 rounds in 1.5-2 hours of fighting.

So in short, yes, there were instances where units would have to make do with low or no ammunition, but it wasn't exceptionally common, as far as I'm aware.