range, load time and the variety of calibers

Are actually all big drawbacks. Plus civilian weapons lack bayonets, which add to a unit's cohesion when entering melee or repelling cavalry, and even as late as the Civil War that's important.

The main disadvantage of a mix of civilian arms is just that the fact that it is a mix and that makes the unit harder for its commanders to control. The unit can't really fire together at any range outside of the point blank conditions shotguns are effective. If they let each militiaman shoot individually at his best range, then fire discipline and the shock value of a well aimed volley go out the window, plus the smoke of the long range shooters will tend to obscure the view of the men with short range weapons.

A Militia unit is going to usually struggle on discipline and cohesion in any case, their lack of experienced officers and soldiers means they are likely to break and run if their battle environment isn't structured very carefully (like putting them behind a fence or something) and that's really hard to do if they don't have a specific effective range when all their weapons will be effective. Militia in general are reluctant to hold their fire to point blank range, and often break once the enemy closes to short range.

In the revolutionary war, and 1812, frontier militia with long rifles were often effective in groups when their officers used them very carefully and were mindful of their limitations. In the Civil War militias with civilian arms were simply lacking in firepower and very prone to panic. In defense of the militia, in early 1861, regular troops were not very much better.