Not a great deal is known about the early linguistic history of Japan, as there was no writing system prior to the arrival of Chinese script in the 7-8th century AD. Prior to that any writing was done in Classical Chinese, which at one time or another has served as a literary language much like Latin throughout Japan, Korea and Vietnam.

The only other language known to have been spoken in the Japanese islands is Ainu, and there is some evidence that it was once spoken outside of Hokkaido. But all we really know is that by the 8th century, Japanese (or the collection of dialects that we call Japanese) was dominant throughout most of Japan. Ainu was confined to Hokkaido, and declined over the subsequent centuries.

The ancestral language to Japanese likely became dominant around 300 BC, when bearers of the Yayoi culture migrated to Japan from Korea and brought with them rice paddy agriculture and both bronze and iron. Because of their technological superiority they gradually displaced the earlier Jomon culture, who were the ancestors of modern Ainu people of Hokkaido, who speak a language isolate unrelated to Japanese.

The Yayoi people settled the south of Japan first, and gradually forced them north. In the Kanto region, where modern Tokio is located, the ancestors of Ainu - called Emishi by the Japanese - resisted until 7th-10th centuries. Gradually they were destroyed on Honshu, but on Hokkaido they remained until the 19th century, when the northern island was thoroughly colonized by the Japanese. Today only a handful of pure-blooded Ainu remain and the Ainu language is all but extinct, altough there are attempts to revive it.