Jamaican R&B (to use an umbrella term for all the genres you mention above) came to be so fruitful thanks to a fair few different factors - many of which you've mentioned above. I'd say that there's three very big social factors playing a role in how it came to be.

Firstly, Jamaica was a British colony with a long history of slavery (/u/sowser discusses the abolition of slavery in the British empire with a Caribbean emphasis here). While British slavery might have stopped a generation before American slavery, it unsurprisingly had lasting cultural impacts. Lloyd Bradley in Bass Culture argues, for example, that Rastafari is more than anything a cultural response to living in a black majority state with a British upper class. In Jamaica, the way cultural capital worked was to encourage an emulation of the white upper class. In contrast, Rastafari was to some extent an explicit rejection of high society's manners - the dreadlocks that Rastafari wore were a symbol of African heritage, for starters, and Rastafari attitudes and behaviours were in opposition to the servile and self-serving attitudes encouraged by interaction with the British upper class.

The history of oppression of black people also meant that (poor) Jamaicans (of African heritage) shared several cultural touchstones with African-Americans, and gravitated towards their music. Jamaican R&B starts, primarily, as a scene simply playing American R&B records. There are later traces of this in how Bob Marley would later cover 'People Get Ready' by the American soul act Curtis Mayfield & The Impressions (or check out the compilation on the Soul Jazz label, Studio One Soul, where artists recording for the Jamaican-based Studio One label cover the likes of 'Time Is Tight', 'Respect' and 'Express Yourself'.) But at the start of the Jamaican R&B scene, the aim was simple emulation of American R&B.

Secondly, because there was an entire lack of infrastructure willing to provide a space for R&B despite its popularity (whether because of simple poverty or other factors), a scene based around portable 'sound systems' developed, where entrepreneurs put together powerful portable PA systems and carried them around to different areas of the Kingston slums (charging for admission and selling drinks).

Different sound system owners prided themselves on their ability to find the obscure 1950s American R&B tracks that had the qualities that Jamaican audiences wanted, often focusing on New Orleans-based R&B with something more of a Caribbean beat (take the 1956 recording of 'My Boy Lollypop' by Barbie Gaye, which I should point out was more New York than New Orleans in origin). As the sound systems grew more prominent, the entrepreneurs behind them started to desire new, homegrown product that better served their particular dancefloors, and the sound system owners therefore branched out into becoming, effectively, record labels.

And if this sound system stuff sounds familiar, one of the reasons would be that Jamaicans in the 1970s like Kool Herc exported this idea to The Bronx in New York, and it became one of the pillars of early hip-hop when sound system culture, toasting, and turntable techniques were applied to funk and soul records - American rather than Jamaican R&B; the famous party where Kool Herc (arguably) invented hip-hop was literally the one where he applied the techniques of Jamaican soundsystem culture to the funk and soul records that his New Yorker audience knew.

Thirdly, it's also important to point out that Jamaica remained a British colony until August 6th, 1962 when they declared themselves an independent nation (mind you, it is still part of the Commonwealth - like Australia - and the Queen is still its official head of state). As a nascent nation, Jamaicans became increasingly proud of growing their own musical sounds, and increasingly focused on their own music rather than American or British imports. The Rastafari culture had previously been shunned by respectable Jamaican people anxious not to lose their cultural capital. But post-independence, Rastafari became seen less as troublemakers and more as a culture that was obviously home-grown and Jamaican, something that they might be proud of even if they weren’t Rastafari themselves.

Additionally, the Jamaican R&B scene received an extra financial boost in the mid-to-late 1960s (and beyond) thanks to its success elsewhere. It’s important to remember that hundreds of thousands of Caribbean people immigrated to Britain in the 1950s and 1960s, and Jamaican music served an important role for young Caribbean-British people and often inspired others who came in contact with it, even before the big success of Bob Marley (Paul McCartney’s ska-influenced ‘Ob La Di, Ob La Da’ in the Beatles’ White Album in 1968, populated by characters with typically Caribbean names - Desmond and Molly Jones - shows if nothing else that British fans of American R&B, like McCartney, were often also exposed to and interested in Jamaican R&B).

Millie Small's 'My Boy Lollipop' in 1964, in a ska style (but not that different from Barbie Gaye above), was an international hit. This was thanks partly to the efforts of Chris Blackwell, a (white) Jamaican who transplanted himself to England and started the Island record label that would eventually sign Bob Marley and propel him to international stardom. Similarly, in the late 1960s, the Trojan record label would do a healthy business in releasing Jamaican songs in England. As English consumers generally expected better sound quality, the rise of Trojan led to improvements in the studio technology used, and thus to further changes in the styles of the music, as musicians tried to exploit the possibilities of new increases in sound quality.

All of these factors meant that - just as with American R&B at a similar time period - there was a lot of competition and thus experimentation in the scene, as different acts and producers strained to find that new 'thing' that would set them apart from their competitors on the sound systems of Kingston, or which might serve them to make it big in the UK. This led to the rise of new genres and sounds like ska and rocksteady and reggae, as one person’s new thing became widely imitated (though rocksteady is basically slowed down ska, and reggae is basically rocksteady that’s a bit funkier - given that the context of the rise of reggae is contemporaneous to the rise of funk in the USA - and a bit more ...stoned).

So yes, Jamaicans transformed the American R&B sounds they were hearing (Jamaica not being a million miles away from the US's southern coastline, after all, Southern R&B radio stations with powerful signals could be tuned into in Jamaica) into something distinctively Jamaican which they could then export to the UK and elsewhere in the world.

In terms of what to read, I think Lloyd Bradley's Bass Culture: When Reggae Was King is pretty accessible and well-written, albeit for a popular audience; Bradley was a sound system operator in the UK in the 1970s and has something of a practitioner's view on the scene, but he's also got a good annotated bibliography at the back of the book. There are also some good academic takes on Jamaican music culture in Dick Hebdige's Cut 'N' Mix: Culture, Identity and Caribbean Music (Hebdige being a sociologist/cultural theorist famous for his 1979 book on subcultures, which introduced the term to the world), while Norman Stolzoff's Wake The Town And Tell The People: Dancehall Culture in Jamaica is an ethnography focusing on Jamaican styles (dancehall, obviously) of the 1990s.

Agree. Wow, what a treat.

This was fascinating to read, thank you. Could anyone recommend any reading on the relationship between punk and ska and how that came to be?