Written

Written scripts are independent of spoken language. Historically, Mongolia DID use Chinese writing, specifically Khitan and Jurchen writing systems. These systems were changed due to politics. When one dynasty fell (Liao) and another (Jin) took over, the writing system changed. Why? Well, what's the purpose of writing? So the government can send you tax bills and court notices. Remember, we're talking a thousand years ago, so people were not going to school, so changing the writing system really means that the privileged few - government workers and scholars mainly - were the only ones who really had to learn something new. After the Jin fell (to Ghengis Khan), a native Mongolian script was used. It's now called "Old Mongolian" which was replaced with "New Mongolian" (aka Cyrillic) by the Soviets in 1946. Politics!

Even looking at Korean and Japanese, you can see the same thing. Korea introduced Hangul, a native script, in the mid 15th century, and Japan has Hiragana and Katakana. Of course, they both have Hanja/Kanji, but they have their own native scripts as well.

Spoken

The reason that they are so different is that Mongolians and Chinese are different peoples. Chinese, Koreans, and Japanese are similar peoples.

If you look at human migration patterns, you'll see that Koreans came from China (across the Yellow Sea) and from Manchuria, and the Japanese came from China/Korea, and Manchuria. The Mongols came from Central Asia and from Siberia. They were two distinct groups and they did not mix until fairly late in history. Once they did start to meet/clash/mix, you still run into the same issue mentioned before - literacy. While the upper echelons of society would have needed to learn the language to deal with the new masters, the common folks wouldn't have had to. For a permanent change, one would have to have schools to teach generations, barriers to employment or social standing for the masses would have to be implemented, and sustained. That just didn't happen in either direction - Chinese (Liao, Jin, and to a lesser extent, Ming) to Mongolia or Mongolia (Ghengis Khan, Yuan) to China.

/u/Muleo mentions the Altaic family of languages, and I don't want to get into an argument about it, but the Altaic group is questionable, at best. Grammatically, Korean and Chinese are quite different. Lexically, they share an enormous vocabulary, especially for basic concepts. Japanese and Korean share a similar grammar and lexis. Pronunciation across all three can range from the same to unrecognizable. But if you look at Chinese, Korean, and Japanese, you CAN see the influence of Chinese AND Mongolian on Korean and Japanese, and if you look at the historical path of human migration across the region, you can fit the pieces together.

For language specific information, check out The World's Major Languages by Bernard Comrie. It goes into some detail about the Turkic origins of Mongolian (which are not disputed) as well as the origins of Japanese and Korean (which, as for as Altaic goes, are disputed).