Essentially, when cannon were able to be fitted aboard ships, and tactics changed to allow ships to batter one another without meeting one another -- trying to ram or board another ship in any sort of a sea was a dicey affair, likely to result in both ships being stove in, so most battles occurred inshore. I wrote a longer answer about naval warfare before cannons. To quote from that:

When we see accounts of naval battles, they are universally fought inshore, usually in a bay, an estuary or even in larger rivers, and are usually fought as an auxiliary to a land warfare engagement. There were no missile weapons that could sink ships, and it wasn't possible to fit rams to the ships of the Norse and their imitators, so it was only inshore that grappling and hand-to-hand fighting could take place. Also, inshore communications were quick enough that they could on occasion call for help when a landing occurred, a prerequisite of getting ships together to fight.

Naval warfare in this era has been compared to mounted warfare on land: ships were a way to transport a group of raiders quickly to a destination and achieve strategic surprise. Shallow-draft ships could also carry raiders hundreds of miles into the interior of a country, using rivers for transport.

A scenario might be that a group of raiders has landed at a village, dragging their ships onto the strand, and that a counter-force manages to trap them with beached ships and burn their means of escape; the raiding force could then be hunted down on land. Many of the battles that we know about during this time period seem to take that form (the details of naval battle are very incomplete). Alternatively, there are some accounts of battles on the water between two fleets, where it seems likely that individual ships grappled with one another, with boarding being the decisive factor in their success. We do know that defense against ships in this era required a combination of fixed defenses (bridges, forts) and squadrons stationed at spots where they could quickly be called to respond. The accounts of battles that we do have tend to list a prince and his achievements (Alfred of Wessex went to sea with a "fleet" (sciphere) in 875, fought seven enemy ships and captured one ... etc.)

Now, as to when cannon were first mounted aboard ship -- we don't exactly know. Quoting myself from an earlier answer:

We know that the English "king's ship" All Hallows Cog was equipped with "a certain iron instrument for firing quarrels and lead pellets with powder, for the defence of the ship" (Rodger, Safeguard of the Sea p. 163) in 1338, but that was certainly an antipersonnel weapon not one that was aimed at destroying ships. Guns seem to have continued being very small in Europe for at least a century or so; when Sir Robert Clifford became Master of the Ordnance in 1496, he had a stock of 63 guns and 406 barrels of powder, as compared to 9,253 bows and 27,804 sheaves of arrows. What really brought about a revolution in gunnery warfare was happening in the Mediterranean, where the Venetians started mounting large guns right in the bows of a galley in the 1470s.

Interestingly, it seems to have been Scotland (not a notable naval power) where we first see heavy guns mounted on ships in Europe. James IV mounted several heavy guns in one or more of his ships during his war with Donald Dubh MacDonald beginning in 1504; in 1506 the castle of Stornoway was battered into submission. It seems likely that Stornoway and the castle of Cairn-na-Burgh (surrendered 1504) were both fired upon from the sea; Carin-na-Burgh in particular is isolated a mile from the nearest land. The Michael, built for the Scottish navy beginning in 1506, was designed from the start to carry 12 guns on each side and three bronze basilisks (very heavy cannon).

The Michael was a bit of a white elephant -- she rotted away in France -- but the increasing trend after 1504 was to mount many cannon on board ship, not necessarily large cannons but a variety of sizes (with often the larger cannon at the stern of the ship). This illustration of the Mary Rose may serve to be, uh, illustrative. By the time of the Spanish Armada, which we know many details of, in 1588, the English ships would have carried most of their heavy guns bowed forward, and committed attacks by "charging" the Spanish, firing all their guns at once, then drawing off to reload. Rates of fire were not high; the English averaged about one ball per gun per hour and the Spanish about one per gun per day.

Does this answer your question? I have written several times here about naval tactics in the Age of Sail.