The short answer to this is deceptively simple: in the 17th century, China was considered an empire due to its impressive size, and Edo period Japan employed Chinese stylings but insisted on independence from China. Therefore for early modern Europeans, in a very vague sense, Japan was also ruled by an emperor. The devolution of Edo governance to daimyo lords must have also been a little empire-like to European eyes, as France and Britain were steadily becoming more centralized.
When we start thinking about how these terms carried on to the present day, it gets more complicated, because Japan had two national rulers in the Edo period: the tennō and the shogun. Westerners more frequently described the shogun as Japan's emperor, while the tennō was not very widely known, and as he became more known he was often called "mikado" (after a Japanese deferential term) and his role in government remained unclear. When Commodore Matthew Perry arrived in Japan on a mission from the United States to forcibly open Japan's ports in 1854, he read aloud a letter from President Millard Fillmore addressed to the "Emperor", by which he meant the shogun.
Why was the term "emperor" preserved in Western languages after the position of shogun was eliminated in 1868? This will require an even more complicated level of detail. To real Japan hands like Engelbert Kaempfer (1651-1716), both of Japan's rulers were called emperors, or Kaisers: Kaempfer called the shogun the Kaiser, but the tennō was the "ecclesiastical hereditary Kaiser." So there was some precedent for using the term Kaiser/emperor in European languages to refer to the tennō in 1868. Japan's unsteady new government wanted to take advantage of such terminology to show that the country was under the control of a very powerful monarch rather than a weak hereditary priest. (They had a completely different strategy for legitimizing the tennō within their borders, which was incredibly complex and way beyond what I could describe here.) As you correctly say, Japan didn't have an "empire" until the 1890s when they began colonizing Formosa.
Incidentally, tennō 天皇 is a unique term in East Asian languages and does not correspond to more generic Chinese terms for "emperor". The word is ancient. There are several theories about where the word tennō came from: (1) the early Japanese emperors took the name of the legendary Chinese king Tiānhuáng/"Heavenly Sovereign" to emphasize their Shinto connection to the heavens; (2) they borrowed a Daoist term to emphasize their cosmic centrality; or (3) they simply applied appropriate-looking Chinese characters to a previously existing native term, sumera-mikoto. After 1868, it was initially unclear whether Japan's government authorities would be using tennō as a generic translation of all emperors around the world, but they eventually decided it would be a unique word that applied to the Japanese sovereign alone. In an explicit rejection of the choice to describe Japan's monarchs as imperial, mass media in Japan's former colony South Korea generally reject the term tennō and refer to the Japanese sovereign as the "King of Japan" 일왕/日王/irwang.
It's hard to know exactly what you mean by "the Western sense" of the words "imperial" and "empire". It's not clear if you're talking about the Roman Empire or the British Empire, to use two quite different examples.
But before looking at that, let's look at the title in Japanese. There were a number of terms for rulers in Japan, such as kimi 君, opokimi, hi no miku 日御子/霊御子 and sumera mikoto but the one that gets translated as emperor is the the title introduced in the 7th century: tennō 天皇.
This title is very clearly borrowed at least in part from China. Rulers in pre-imperial China were called wang 王, a title that is conventionally translated into English as "king". During the Zhou dynasty, the Zhou king ruled "All Under Heaven" (tianxia 天下), that is, over vassal states that paid tribute to the Zhou, but were otherwise autonomous. As time went on, these states became powerful in their own right and their rulers began to arrogate the title "king" for themselves. Gradually the Zhou ruled in name only, and states battled for supremacy in a chaotic period called the Warring States.
Out of these wars, the state of Qin was ultimately victorious, and the ruler of Qin, Ying Zheng, set out to unite the various states under his control and abolish the vassal system. He called himself Qin Shi Huangdi, the title Huangdi being two words for higher beings put together to create a new title. One of the words is often translated into English using the neologism "thearch" huang 皇, and the other also (somewhat confusingly) gets translated as "emperor" di 帝. This new title (huangdi 皇帝) became the standard title for the supreme ruler of what we now call China. Because Qin Shi Huang is regarded as uniting the territory under one rule — standardising all kinds of things like laws, the script, roads, weights and measures — he is regarded as ruling a united territory that befits being called an empire. While this is a quite simplified version of what happened (how much Qin Shi Huang really did standardise and unite is still debated) scholars generally agree that the change occurred.
The Japanese term tennō is not quite the same as huangdi — it only uses the huang of huangdi. But it is nonetheless clearly Chinese derived. The ten of tennō means heaven or sky, and reflects the Chinese concept of the Son of Heaven tianzi 天子, a common term for the highest ruler which pre-dated the Qin, and which illustrates his (and sometimes her) status as an intermediary between the realms of heaven and earth. According to the semi-legendary accounts in the eighth century chronicles Kojiki and the Nhon Shoki, Yamato rulers had used the title tennō since the first human emperor Jinmu. This is just a legend however, and it's not clear exactly when Japanese rulers began using the term, a situation not helped by the scarcity of written records from before the late seventh century. The Chinese Book of the Sui however does suggest that Japanese (or wo 倭 as they called them) rulers were called "Son of Heaven", and earlier memorials to China refer to Japanese rulers as "king". Whenever it first was used, by the late seventh century tennō appears to be solidly established.
So why "emperor"? Tennō could and is sometimes rendered as "heavenly sovereign", but there are two reasons why it gets translated as emperor. The first is the Chinese connection which has been explained above. Not only did Japanese rulers adopt Sinic terms for themselves, they also quite early on claimed to rule "All Under Heaven", a clear borrowing of the Chinese concept. The second reason is that later Japanese rulers also used the term tennō, deliberately harking back to the ancient and semi-legendary past. When Japan became the last country to become a world power in the 19th century sense under the Meiji restoration, they very much were an empire in the way the British Empire was an empire, so it makes sense to translate the title of their monarch as "emperor". Japan is no longer an imperial power in that sense, but the monarch is still there with the same title, so it makes sense to continue translating tennō as "emperor".
Sources
Duthie, Torquil. Man'yōshū and the Imperial Imagination in Early Japan. Leiden: Brill, 2014.
Kirkland, Russell. "The Sun and the Throne. The Origins of the Royal Descent Myth in Ancient Japan." Numen 44.2 (1997): 109-52.
The special issue of the Japanese Journal of Religious Studies titled “The Emperor System and Religion in Japan”, Japanese Journal of Religious Studies 17.2-3 (1990), especially Joseph Kitagawa’s article, “Some Reflections on Japanese Religion and Its Relationship to the Imperial System”, 129-178.