There were riflemen units in the Napoleonic and American revolution wars but the standard solider had a musket. Was it a matter of production or pride in the musket?
Rifles achieved their increased accuracy by taking a small lead ball (the bullet) and wrapping it in a wad of cloth (usually felt). The felt wad would then grip the spiral grooves of the bore which would in turn "spin" the bullet as it traveled down the barrel, this spin stabilized the ball as it traveled through the air toward its target leading to an increase in accuracy. The increased accuracy of the rifle was an obvious advantage, but not without its consequences. These rifles were more costly to produce and a smaller number could be made owing to the need for specially trained gunsmiths to manufacturer them; tens of thousands of the famous Pennsylvania (Kentucky) long rifle were made, but in that same time millions of the standard "Brown Bess" musket were produced. Second, for a rifle to be effectively employed required specialized training and experience in handling and using the rifle, this was not a weapon for the common infantry man. The rifle was the weapon of choice for hunters, sharpshooters, and Jägers. The rifle holds a special place in the American psyche because it was the weapon for the individual, not the closed ranks of most military formations. There is also the issue of the rifles battlefield performance, trained rifleman could get off roughly two shots (optimistically) in the time it took a man armed with a musket to fire three shots. In close range fighting between lines of infantry volume of fire is more important than individual accuracy. A rifleman would spend considerable time just trying to back the tightly wrapped lead ball into the barrel of his rifle, often times relying on a small mallet to drive it in. Rifles also commonly lacked a bayonet plug on the end of the barrel which would be used to affix...a bayonet. In hand to hand fighting or in repelling cavalry the advantages afforded by the reach of a bayonet and its use by closed formation of infantry was considerable. And over time after repeated firing in battle gunpowder residue would build up in the barrel reducing the accuracy of the rifle until it was cleaned, not something you want to do in the middle of a firefight.
Rifles were not exclusive to the American continent, the Austrians had introduced the Girandoni air rifle in 1780, the Germans had their famous Jäger light infantry, and the British had introduced the Baker rifle in 1800. The Girandoni air rifle was expensive and somewhat difficult to use while the Baker rifle solved many of the problems associated with muzzle loading rifles, it was still only produced in small number owing and used by specially trained soldiers. What was needed was a weapon that could be manufactured cheaply and in great numbers and was easy enough to use that even a common infantry man could do it. Something that often goes overlooked is the issue of battlefield command and control. You'll notice that previously rifles were almost exclusively used by specialist military units (riflemen, sharpshooters, light infantry, etc.) Men were selected for these units because of their technical skills, physical stamina, and self reliance. Light infantry ere expected to operate in rough terrain over great distances away from the main body of the army. But when battle was joined between two armies it would take thousands of officers and non-commissioned officers to effectively maneuver tens or hundreds of thousands of men in tight formations, (individualists need not apply). The battlefield was a chaotic and smokey arena, its why soldiers wore such brightly colored and distinct uniforms. It was hard enough to see what you were firing it or who was firing at you, and near impossible to even see individual targets. The infantry regiment was simplest and easiest method of commanding a few hundred men to move together and ideally all fire at the same target at the same time. Their volume and weight of fire as a massed formation was what gave the infantry their killing punch.
However, the landscape of infantry combat changed dramatically in 1847, the year French officers Claude-Étienne Minié and Henri-Gustave Delvigne introduced their conical-cylindrical bullet with a hollow base, hence known as the "Minié ball". Compared to earlier rifle technology the Minié ball was revolutionary for both its ease of use and economics. The Minié ball would be loaded into the muzzle of a rifle base first (pointy end up) and when fired the rapidly expanding and super heated gasses of the powder charge would fill and expand the base of the Minié ball allowing it to then grip the grooves the rifling. Minié balls were easy and cheap to manufacture in tremendous numbers, they were simply cast lead. The industrial revolution increasingly made the manufacturing of rifles cheap and easy, and even muskets were converted into rifles by milling grooves into their barrels. The Minié ball was also easy to use for even the common infantry soldier, it could be loaded into the muzzle of his rifle and rammed down the barrel in much the same way a musket would be loaded.
One piece of the puzzle was ammunition design. Until the introduction of bullets like the Minie Ball, loading a rifled musket was a difficult, time-consuming endeavor. With the round ball that was standard for ammunition at the time, the ball was close to the same size as the bore, with wadding used to provide a tighter seal and help hold the ball in the gun until fired. Unfortunately, for rifling to work, it needs a tighter fit between bore and ball to allow for the rifling to engage the ball effectively.
Modern rifling gives some idea of how tight you need the seal to be for the best effect. American 30-06 M2 Ball of WW1 vintage had bullets 0.3086 inches in diameter. Standard bores, however, were .300 inches in diameter, while the rifling grooves were about 0.005 inches deep. The result is that the bullet ends up being formed to the rifling, and on fired bullets the imprint of the rifling is generally very clear on the bullet.
Going back to muzzle-loaders, the problem becomes pretty apparent. Rifled muskets pre-Minie Ball didn't have as tight a fit to the bore as modern rifles, but they still used a significantly tighter fit than a standard smoothbore and thus became significantly more difficult to load. That translated to slower rates of fire and ultimately resulted in something that was found to be unsatisfactory for standard line infantry.
That's where the Minie Ball comes in. The Minie Ball and systems like it made muzzle-loading rifles practical by being small enough to fit down the bore easily like on a smoothbore and expanding upon firing to engage the rifling. For the Minie Ball, this was accomplished by a cavity in the back of the bullet that would be forced outwards by the pressure of the expanding gases. Though not perfect by modern standards, it finally provided armies with a bullet that could be loaded as easily as an old smoothbore musket but work well with rifling. These new types of ammunition also afforded some other beneficial side effects, including higher velocities (thanks to a better gas seal) and longer effective ranges thanks to not only the use of rifling, but their more aerodynamic shape. These types of ammunition were developed in the 1830s and 1840s, and relatively soon afterwards is when we see armies all over the world adopting rifled muskets like the Pattern 1853 Enfield and Springfield Model 1855.