For a partial answer, there is useful information in La Force, J. (1969). "The Supply of Muskets and Spain's War of Independence". Business History Review, 43(4), 523-544. doi:10.2307/3112040

La Force mostly deals with the production of muskets c. 1800. Notably, La Force gives a contemporary estimate that it would take a factory with 2265 workers, including 690 master gunsmiths, to produce 300 guns per day.

La Force notes that the city of Ripoll, in Catalonia, had been capable of producing 300 guns per week in the mid-16th century, with a workforce including 80 master gunsmiths. If each master had 2 other workers (similar to the estimate above, with masters being about 1/3 of the total workforce), then their productivity was similar to (and a little higher) than the expected productivity in 1800.

From these figures, we can estimate that it would take a master gunsmith and 2 other workers 2 days to make a gun. This doesn't quite answer your question, since we don't know the length of their working day. If we assume 8 hours of work per day, then this is approximately 50 hours of work per gun.

For a somewhat later figure, the newspaper Daily Constitutionalist (Augusta, Georgia), August 22, 1862, p. 1, c. 5, notes of one factory:

They will employ about 200 hands and make fifty muskets per day.

which is about double the Spanish productivity above. This is likely the benefit of increased mechanisation (e.g., machinery for rolling barrels, making stocks).

"Having bought everything I need from the local smith" implies that the gunsmith's shop is not making the lock, barrel and furniture ( the other metal pieces, like the buttplate and triggerguard), that it is instead stocking the musket- shaping the gunstock, assembling all those metal pieces into it, then putting on a finish. Some gunsmiths could and would make all the parts of a gun in their shops: they might have a big shop with a diversity of talents, have barrel forgers, workmen on the machine for boring the barrel tubes, others to file/grind the outside profile of it. There would be lock filers, who would take the rough-forged lock parts and file them to fit, polish the parts and harden them. There might be a small foundry set up to cast the brass buttplates, triggerguards.

But if the gunsmith was in a small town, he might not be able to keep a very large shop profitable with only local business, so he might have a smaller shop and few workmen. If he contracted to furnish a militia, then, he might be buying locks , barrels and even furniture and just doing the stocking. That was possible. Jost Amman had split gunsmiths and gunstockers into the two trades by the mid 16th c. and the British would indeed move to a kind of "putting-out" system for their muskets in the 18th c. that split up the tasks. The Board of Ordnance would purchase the metal parts- at least, the locks and barrels-from some large barrel and lock-making shops ( mostly in Birmingham) and stockpile them at the Tower of London. These would then be furnished to contract shops to stock muskets. They'd get the metal parts along with a musket to be used for making patterns, so that the end product would be uniform ( and, so that a commanding officer could not simply take the money allotted to him for furnishing his men with good muskets and buy junk ones, and pocket the difference- a problem in the 17th c.) So, your idea is plausible.

And that figure of 2 days for 2 men to produce a musket in 1800 in the Pyrenees, that wotan-weevil cites, is something within reason for the time need for stocking a musket. But, just to be clear, it would not be anywhere close to enough time to make the locks, barrels and furniture- with those, there'd be more than 200 man-hours involved, not 50. And so , with all that labor, you should also bear in mind the cost. In 1775 ,in the heated beginning of the American Revolution, a militia musket in Massachusetts went for $12.50. A farm worker there might get $1.75 per week- and that's when he was able to be working. So, maybe around a fifth to a sixth of his yearly income. That 17th c. musket was going to be similarly expensive. If I was a gunsmith in that small town and was asked to supply, say, 30 muskets, I'd first want to know just who was going to be paying for them and would likely want a substantial down payment. I would be laying down a lot of my own money for the metal parts, and expecting a lot of money for the finished pieces

M.L. Brown:Firearms in Colonial America