So, to preface this: as with many things related to etymology, there is no concrete evidence to how this word developed.

There are, however, widely accepted theories.

The word "Japan" is a phonetic mutation based off of Chinese pronunciations of 日本.

The first European to mention Japan in detail is Marco Polo. However, Marco Polo never actually visited Japan, and instead learned about it in China. His understanding of Japan was very skewed and mythical because it had all been tales from mainlanders, who themselves have barely been to Japan. The Chinese did not see Japan as a land worth visiting, and so their understanding of it was very poor. There were a lot of fabricated stories such as the sun originating on the islands, the people being exceedingly dwarfish, or the presence of a mythical tree called Fusou. These myths would get transferred to the mongols and later Marco Polo himself. For example, he describes Japan as a land where palaces are made completely out of gold and pearl. Whilst this is true for some of the earlier Fujiwara temples, the Chinese and Marco Polo exaggerate it to the point where the archipelago becomes an eastern El Dorado.

The name he used for this mythical wonderland is Cipeng, which naturally was also derived from Chinese.

日本 in Chinese can be pronounced as Ziben, Zipeng, Riben, depending on regional accents and dialects. Cipeng may have come from a southern dialect.

Most Europeans would know Japan through Marco Polo, and would thus know it as Cipeng or Cipang. For example, Christopher Columbus, who was greatly inspired by Marco polo, referred to Japan as such. Cipeng could also be written as Zipeng, Jipeng and Gipeng as there was no real standard. The most popular modern spelling of Cipeng is Zipang.

The predominance of using the "J" sound came from the Portuguese, who had extensive trade and colonization in Southeast Asia during the 16th Century. Also derived from Southern Chinese dialects, the Southeast Asians referred to the island nation as Japun. Because of this, Portuguese Jesuits pronounced Cipeng as Jiapeng or Jiapang.

The Portuguese were the earliest Europeans to establish contact with Japan, when they landed on Tanegashima in 1542. They sent Jesuits to the new heathen lands in order to convert the local populace. This was very successful, as the Japanese were in a time of civil war, reforms and revolution and the new foreign knowledge and insight that these "Nanban" brought greatly stimulated the warring states. Portugal established monopoly over trade with Japan until they were supplanted by the Dutch under Tokugawa Ieyasu. For much of the 16th Century, where the West truly started developing an understanding of Japan, Portugal was the premier expert on all things relating to it. The earliest recorded spelling of Japan in English is "Giapan", translated from a Jesuit letter. Every European name for Japan from the Italian Giaponne to the English Japan would be codified from these Jesuit accounts.