Interesting thing about the streltsy. Some believe that the longer streltsy arms (poleaxes, bardiche) may have been used to support the musket as they fired it. The elite Ottoman Janissary soldiers adopted muskets rapidly, using them in company with weapons like swords, axes, or in case of palace guards, halberds. The Highland Scots developed a strategy of having men with muskets firing at once, then charging while giving a battle cry. They dropped the muskets, crouched in the smoke (to make chances of a counter volley hitting lower) and rushed to attack the enemy, to exploit the shock of the musket volley. This tactic seems awfully dependent on successfully pushing the enemy off the field though, because you loose lots of military hardware if it fails. For another example of how muskets were integrated, take the Caroleans of Sweden (here I go, cementing myself as a Sweeaboo) who utilized a tactic called Ga-Pa, where they would march or run up to the enemy, fire at very close range, and then close in with rapiers (the average infantry man was armed with a musket and rapier, men on the edges of the formations, the grenadiers, had bayonets in order to fend off cavalry) and one source suggested that they tucked the gun under their arm and then charged with the sword, but I am not entirely sure what they did with the guns once fired.

Muskets were above all, battlefield weapons. You wouldn't carry them around in places other than a battlefield (why do the three musketeers carry swords? Muskets are bulky and not very glamorous, at least compared to skilled swordfights) the same way you wouldn't carry a pike around. You would keep something like a sword (in the musketeer's case) as a sort of sidearm, a defensive weapon in cases where the enemy got too close for the musket to be effective. However, it's also pretty easy to overestimate how much these weapons weighed. A sword or pike that wasn't too long would have weighed only a few pounds, and while muskets usually fell a little higher, sometimes over 10 pounds, they're not impossible to carry.

Another possibility is the sling, some sort of rope or piece of leather to allow the weapon to be carried over the soldier. Grenadiers were provided with slings in some cases, as to allow them to sling the gun over their soldier to throw grenades. Some muskets might have a metal ring further up the weapon to allow the attachment of a sling. I'm thinking that slings may be the solution in some cases. Not all, but some. I would figure you would loose some of the use for a sling if the only weapon you needed to carry was a gun and attached bayonet, as you had no need to change to any other arm.