Short answer is not too often; the guns could be deadly at close range and useless at long range, but it did improve over time. The guns were grossly inaccurate for many reasons: the lack of any serious naval gunnery training (which would only appear a bit before the 20th century), the lack of standardization in making the guns, the lack of sights, different gun crews for each guns and ship motion greatly affecting accuracy. The solution to this was to fight in lines of battle which would often lead to inconclusive results. To grossly simplify what that is, you'd have the two fleets form lines next to each other at very close range to maximize accuracy and fire broadsides in order to maximize firepower. Ships powerful enough to be in the line of battle would be called a ship of the line, the precursor to the battleship. As mentioned earlier, a series of inconclusive battles forcibly made admirals look elsewhere, the most known example would be Nelson's maneuver during the Battle of Trafalgar where he met the Franco-Spanish line of battle head-on and obliterated them. Essentially, deadly at close range (25-50 yards), but useless at long range. Not until the middle of the 18th century would the gun captain be able to shoot, aim and decide when to shoot thanks to the introduction of the gunlocks. And not until the end of the 19th century would gunnery be taken seriously for real

Sources:

Mahan - The Influence of Sea Power upon History 1660-1783

Massie - Dreadnought

Gardiner - The Line of Battle: The Sailing Warship 1650-1840