The ball is a loose fit at first. It can move around in the barrel. If the powder goes off and the ball is not firmly on top of it, is just a little in front or midway, bad things can happen: like the barrel swelling. So, yes, at first the wadding would be rammed down on top of the ball.

Black powder is dirty stuff, and after the fouling has built up, the ball is a tighter fit and the wadding is less important. But it takes the same amount of time to ram down the ball with or without the wadding, and there was a drill for loading and the soldiers would be trained to follow it. For good reason: a muzzle-loading musket could be rendered inoperative by improper loading. If the recruit ran the ball down first, before the powder, he could only bring it to an officer who could extract the ball with a ball-puller, and that would take time. If he ran the wadding down before the powder, the gun might not go off and , again, the charge would have to be extracted. Careful adherence to the drill meant this would not happen. Or, at least, was much less likely.