They're actually not completely different as you've suggested.

Nihon (にほん) and Japan are actually the same word, just pronounced differently thanks to transmission from one language to another. Quoting from Wiktionary cuz it's faster: "From Dutch Japan or Portuguese Japão, from Malay Jepang, from Sinitic 日本".

日本 in Old Chinese it was *nit-pən (pən kinda like English "pun") which you see in today's Nippon, an alternate reading to Nihon. The sounds /p/ and /h/ are variations on a theme, and you'll find them alliterating throughout the spelling of Japanese (e.g. shin-bashi/shin-hashi 新橋). As for the /ni/ and "ja" thing, that's also a very common historical variation. Many words that were once beginning /ɲ/ (think English n in "onion") in Old/Middle Chinese are now /ʒ/ (think French j as in "Jean") in modern dialects. And of course in English the j sound (dʒ) and /ʒ/ are commonly swapped, which is why nearly every single news anchor on American tv says Beijing as Bei[ʒ]ing instead of Bei[dʒ]ing like it should be in Mandarin.

In Malay, "Jepang" is again just another version of 日本, which in modern Mandarin is /ʐɨ.pən/, really not so far off as you'll probably agree.

So in the case of Japan it's really just a matter of how different languages have realised this word, and then other languages borrowing from them, until eventually you end up with things as different as Japan and Nippon, but in the end they're ultimately the same thing.

China's a bit more complicated. The most widely accepted etymology for the English name is that it's a variation on "Qin-y", Qin being the first major unifying dynasty, -i being an adjectival marker. In Persian čīn-ī (چینی) meant "from Qin" and referred to the sort of porcelain you might now hear being called "china" in English. Old forms of Persian is where Europe got the name, but Persian got it from Sanskrit cīna चीन (/tʃi:na/). Still most people will agree it ultimately derives from the name of the Qin dynasty 秦朝.

As an added note theres the phrase in Mandarin zhina 支那 (zhi like "jur" in "jury") which was a Japanese derogatory term for China, a sort of retroactive version of "China", but this is considered significantly offensive today, and you'll really only see it showing up on Japanese maps and references from the first half of the 20th century.

Also, Zhōnghuá isn't the name of the modern countries. The People's Republic of China is Zhōnghuá Rénmín Gònghéguó 中華人民共和國 and Taiwan (Republic of China) is Zhōnghuá Mínguó 中華民國. The "Zhōnghuá" part is a sort of regional nickname which you can consider meaning "central enlightenment" or "central culture" and predates either of the modern countries taking the name "China". The more common name for China is "Zhōngguó", but this too has a very complicated history and is not simply "middle kingdom" as some will surely suggest.

Just in case it comes up, the term "Middle Kingdom" is also a more recent appellation, and in it's original sense the Chinese term did not actually carry that meaning. Regarding "Zhonghua":

During the Western Zhou 西周 (1045–771 BC), the term Zhongguo referred to all ‘inside the kingdom’ (guozhong 國中 in the sense of guonei 國内). Afterwards the term was used to refer to some of the feudal states in the middle and lower Huang He (Yellow River) region. In the Confucian classics the term Zhongguo represents a “concept to differentiate the Huaxia [華夏, literally ‘glorious and extensive’ [which] originally referred to a group of people living along the Yellow River] from the barbarians.” During the third and fourth centuries AD, the terms Zhongguo and Huaxia were abbreviated to Zhonghua 中華, an expression that soon came into general use.

That's quoted from Lehner, Georg 2011, China in European Encyclopaedias, 1700-1850. The modern name 中國 Zhongguo is an abbreviation of a longer phrase, and the sense of "middle kingdom" should actually historically be translated as "central kingdoms" or "central part of the empire" and never actually referred to the whole of "China" until much later, much more recently. See this thread for more discussion on that point.

edit: thanks to /u/Hanfresco for a typo correction