Maybe I am overestimating the variance in mutual intelligibility for different dialects, but my understanding is that even within the same country, speakers of German, Italian, and Chinese (Mandarin vs. Cantonese of course) have trouble understanding each other.
Within the Arabic world and Spanish-speaking world I have heard of similar difficulties across countries, even if there is a "standardized" version that is taught with schools.
Sure there are differences in slangs and accents across the English-speaking world. But for the most part, a British, American, Australian etc. person has relatively little trouble understanding the others. Am I reading into this wrong, or is there some special reason why English didn't split off into separate "dialects".
For one, you're overestimating mutual intelligibility. A number of Americans would have difficulty understanding people like Billy Connolly. Of course there is no way to objectively quantify mutual intelligibility between language varieties to begin with, but I think you're missing some of the variety in English. Take /u/ali_421's point about dialects in the UK which you may simply not know exist.
Chinese has also had a lot longer time in China to diversify than English has in North America, and a much larger place to do it than English in Europe, though in both places there's been significant numbers of other languages to get influence back and fourth. That's true of Arabic as well.
However if you really want you Chinese example to be more appropriate, take Mandarin dialects to other Mandarin dialects and look at intelligibility there, not Mandarin to Cantonese. Mandarin to Cantonese is between languages like French and Portuguese or between English and German. "Chinese" is the language family as "Germanic" is to English, if you want things to be on a more level playing field. So then I have very little difficulty understanding different dialects of Mandarin since I'm familiar with the differences, the same as I am with English.
But as everyone else has said already, it is completely subjective.
This is somewhat subjective really, but in the UK or even just England it can be very hard to understand the accent of someone from a different area if it is very dense. A thick scouse, brummie/black country, or geordie accent can be really impenetrable to someone from elsewhere in the country.
I have lived 50 miles away from Birmingham most of my life (and in the city itself for a good period of time) but I have to really concentrate and ask for things to be said again when talking to someone with a really strong Brummie/Black Country accent.
Then of course when we change countries it gets even more noticeable in some cases. Cheryl Cole possibly being kicked off the US x-factor as the American audience apparently couldn't understand her (very mild) Geordie accent. Or even more recently the complaints of Peter Capaldi's Scottish accent confusing people in the US. Being two recent examples (not exactly history but shows how the barriers are still there).
It's subjective, but Jamaican English is quite different than many other English spoken around the world. Here's an interview with Bob Marley, there's a voiceover in the beginning saying his dialect can be difficult to understand.
Question for clarification: are you allowing for colloquialisms and slang that certain English speakers would have compared to others?
I'm not sure that if they were incomprehensible they would count as dialects.
Mandarin and Cantonese are not dialects, they are languages. The difference between dialects of Spanish ranges considerably, and can cause difficulties in communication, but they are still mostly mutually intelligible, especially if the speakers make an effort to avoid localisations.
I think I would have to say that your definition of a dialect is overly narrow from a linguistics point of view. Indeed, one standard rule for determining if something is a dialect is mutual intelligibility between two (dialect) speakers.
Oh buddy der I tells ya my son you stays where ya too I comes where ya at and gives you a right good story der wha. See we's got dis whole ting about hows we speak in de places der right? And you wants ta know how we does it so same like, but you gets two b'ys form de bay getting all talky wit each udder and tries to write it down, you gots nuttin' on dem der eh? And not jus dat, but der speakin is right slippy til you comes up and ask dem "What're ya at", and dell right quick say "Oh, sorry, we were just speaking like the folks at home."
This is an only slightly over-exaggerated example of a couple of a Newfoundlander telling you a story, if you're a fellow Newfoundlander. Similarly, put two people from Cape Breton together, and they'll speak entirely differently than they will when speaking with someone from the U.S., Britain, or even the rest of Nova Scotia. Newfoundlanders are well known in Canada for have their own, distinct, and difficult to understand dialect, but these days most are exposed to a broader range of the English-speaking world than the surrounding outports, so they learn to speak a couple of different ways.
I can tell you from experience, I've also seen people from Cape Breton switch from no distinguishable accent to unintelligible in a few moments; similarly, Acadians from the south shore of Nova Scotia can do the same (though that case is incorporating French, as well).
In short, people in the same country, and even the same province of the English-speaking world can be unintelligible to each other, and English has been split into different dialects.
People have compiled and entire dictionary of Newfoundland English, with some of the specific words used. So, while you may not have come across these dialects yet, they do exist.
There definitely are dialects of English that are quite unintelligible to a speaker of any of the Standard Englishes. Possibly the most major of these dialects would be Lowland Scots. I don't know if this is an actual example of bona fide Scots, but this video shows how incomprehensible Scottish Neds can be to an outsider.
It's been a while, but having taken some linguistics courses in undergrad, one phrase always stuck with me:
"A language is a dialect with an army and a navy."