The 'pirate' accent is an exaggeration of what could be roughly called a 'Wessex' accent in South Western England. The most famous strand of this dialect is Bristolian, but it's similar to accents found in Gloucestershire Somerset, Devon, Cornwall and Dorset. For reference, consider the array of accents on display in Hot Fuzz; it's not an exaggeration to say that to many of us from the region, David Bradley's farmer character is actually perfectly intelligible without the comedy translation. The stereotypical 'pirate' accent was famed by the actor Robert Newton, from Shaftesbury in Dorset. The Dorset accent is essentially slower, broader version of the Bristolian accent. Newton played Long John Silver in two movies in 1950 and 1954, as well as purportedly Bristolian pirate Blackbeard in 1952. The success of these films meant that Newton's Dorset accent became widely associated with pirates.

Bristol today remains a major port city, but in the 16th-18th centuries it was one of the most important commercial ports in Britain and a nexus for a major portion of trade between Britain and the New World. This naturally meant that a lot of sailors - and by extension some pirates - are likely to have spoken with a Bristolian accent. Newton's Dorset accent being broadly similar to this helps compound the myth.

Where did the whole Pirate 'Yarr' thing come from?

It's a feature of English spoken in southwestern England. Many of the English pirates came from this area of the UK.

And did pirates actually have that gruff accent?

The closest modern-day equivalent to pirates happen to be Somali nationals. These tend not to have west country accents.