Japanese uses a lot of hard consonants compared to a language like Korean, and it just sounds very different. A pure syllabary would never work in a language like Korean, as elegant of a writing system as Hangul is. But it works fine for Japanese. When and how did these languages diverge so much?
Like one could say it is because they are more isolated because they are an island; but so is Britain, and English and German still share a lot more similarities than Japanese and most other East Asian languages.
And it also isn't exactly like there wasn't a lot of cultural exchange happening with China, considering that Kanji are just Chinese characters that the Japanese adopted.
So there are a few things to unpack here.
First, geographical proximity does not necessarily indicate linguistic similarity. You get similarity between two languages through one of two ways: 1) phylogenetic relationship, and 2) language contact. Let's break these down.
This term is borrowed from biology, and used to indicate that two (or more) languages descend from a common ancestor. The more closely they are related, the more similarities they are likely to have. This should be intuitive: if the languages haven't had enough time to evolve on their own, they will still be quite similar to their ancestor, and also to each other. If you look at Spanish, Portuguese, and Italian, for example, it's not that difficult to see the resemblances between them, although the differences are also quite obvious.
On the other hand, if two languages are related, but diverged a long time ago, their relatedness will be obscured by a bunch of different changes. Take, for example, English, Greek, Albanian, and Hindi. All are Indo-European languages, and so are related on some level, but a non-expert will be very hard-pressed to say that there is any relationship between them.
If a group of languages have been shown to be related, we put them together into a language family.
Phylogenetic relatedness isn't the only kind of relationship languages can have. Languages that are in contact with each other will often borrow words and phrases, and sometimes even phonemes (sounds) and grammatical structure.
Note that contact is not always symmetric. Linguistic communities with great cultural influence will 'export' loanwords into surrounding languages, but will not necessarily borrow any words from them.
English borrowed a metric shitton of words from Latin, French, and Greek. So much so, in fact, that many laypeople erroneously assume that it is more closely related to French than to German. But if it has so many loanwords, how can we tell that it's in fact a Germanic language, and not a Romance one? By using the basic vocabulary.
The basic vocabulary is a short list of the most basic terms common to all languages of the world. The first ever basic vocabulary lists were created by Morris Swadesh, and are called Swadesh lists. They come in several versions, and most are 100 or 200 words long. More recently, researchers at the Max Planck institute created a new basic vocabulary list called the Leipzig-Jakarta list, that is comprised of words most resistant to borrowing.
The point of these lists is that the most basic words are used every day and are therefore less likely to be loaned from other languages (though less likely does not mean impossible!). We can then use these lists to compare languages, and hopefully sift out many of the loans.
Of course, we can't just compare languages willy-nilly. We need a framework, and one exists just for this purpose: the comparative method.
The comparative method is easy to learn, but difficult to master. It's a simply but incredibly powerful tool that lets us look into languages' past. The main axiom of the comparative method is the Neogrammarian Hypothesis: simply put, that sound change is regular and systematic.
Thus, when we try to show a phylogenetic relationship between two languages, we can't just grab a few similar-looking words and call it a day. Instead, we have to draw up sound correspondences that account for ALL the sounds in ALL the words. That can be very time-consuming.
Your question assumes that Japanese is related to surrounding languages, but in fact no such relationship has been proved. There were attempts to connect Japanese to Korean (mostly under the umbrella of Altaic), but these were examined in detail by Alexander Vovin in numerous articles, and summarily dismissed due to faulty methodology. So simply put, Japanese isn't phylogenetically related to Korean or Chinese.
However, Japanese did in fact have very intense contact with Chinese for a long period of time (so did Korean btw). Japanese borrowed a lot of vocabulary: so much of it, that we can actually divide it into strata based on the time period and the dialect from which these words were borrowed. That might give the impression that there is some kind of genetic relationship between the languages of East Asia, but no, it's all contact as far as we can tell.
A lot of people confuse writing systems with the languages themselves. Writing systems and spoken languages are orthogonal to each other. Whatever writing system a language uses has absolutely no bearing on its phylogenetic relationships.
You did say that a pure syllabary like Japanese kana wouldn't work for Korean (not without some serious adaptations at least!), and you're right. The reason for this is that Japanese and Korean have a different syllable structure. Japanese syllables can be modelled as CV(N), that is, a consonant followed by a vowel, optionally followed by a nasal (-n). Korean allows more types of syllables, though its structure is still quite restricted compared to that of English.
Note that syllable structure is not necessarily indicative of phylogenetic relationship. Languages in the same family can have radically different syllable structures, and "sound" very different.
So in the case of Japanese, the differences are due to the fact that it's completely unrelated to Korean or Chinese. However, even related languages can sound very different from each other due to accumulated sound changes, or language contact.