They fulfilled very different roles. Let's take two extremes of potential targets - a bomber flying around 200mph at 25,000 feet, and a fighter diving to strafe a ground target, ending up under 1,000 feet at a speed of 400mph or more.

For the bomber, heavy flak (generally 75mm and above) is needed as smaller shells won't even reach it - light flak is typically effective up to around 5,000 feet. You also need to work out where to shoot - in the 20 seconds it takes a shell to get to 25,000 feet the bomber will have travelled over a mile. Heavy flak was used in conjunction with mechanical or electro-mechanical analogue computers called directors (in US service) or predictors (in UK service) that predicted the position of an aircraft and directed anti-aircraft guns accordingly. They were supplied with inputs such as range and height from other instruments; in this picture of a British 3.7" AA battery you can see a gun in the background and predictor in the foreground with stereoscopic height and range finders behind it. The shells had timed fuses, the settings calculated to explode at the height of the target, exploding in a cloud of fragments so a direct hit was not necessary.

For the fighter, heavy guns are just too cumbersome to traverse and elevate at the speeds involved, and there isn't time for complex calculations to work out where to shoot. Light flak (generally 40mm and smaller automatic weapons) could adjust more quickly with either a simpler director/predictor or manual aim, using the high rate of fire to throw out more shells for a greater chance of a direct hit.

All nations therefore employed both heavy and light anti-aircraft units as appropriate, the former often defending strategic targets (though also useful against ground targets, the German 8.8cm gun being the archetypal dual use weapon), the latter generally more mobile to accompany ground forces, and used around targets more likely to be attacked at low level such as airfields or V-1 launch sites.