We should definitely wait for an answer from the specialist in the corresponding field of research (ancient Japan), but I can at least provide a few clues to consider your question.

1: The mean height of ancient Japanese is probably taller than generally assumed

A Classic study on historical osteology in Japan (Hiramoto 1972) estimates that the mean height of Japanese male was ca. 163 cm (under the heading of Kofun Period: Hiramoto 1972: 224). This mean value is more than 5 centimeter higher than that of the estimated mean height of Edo period.

If I remember correctly, the average estimated height of the Scandinavians during the Viking Age (about the contemporary as Nara period in Japan) was ca. 170-175 centimeters, so if you 'teleported' one of the Vikings to Japan in Nara period, he was probably not so taller (about ca. 10 centimeters) than the Japanese in average.

2: You were probably not the only person with blue eyes in Japan, so as long as staying around the capital (Heijo-Kyo, Nara) you would be treated as a foreigner, I assume.

This is one of the famous ritual mask owned by Shoso-In, Nara, called Sui-Ko-Oh, the drunken king of the Persians. It also has artificial mustache and beard made of horse hairs. An example of the replica of this kind of mask is also found here. Some researchers (including Moriyasu 2020 and the author of this official blog of Nara National Research Institute for Cultural Properties prefer the interpretation of his identity as the Iranian Sognians to the Sasanian Persiansm though.

We can confirm in contemporary documentary sources that at least one Sogdian by way of Tang China and two Persians came to Japan and stayed in Nara (Moriyasu 2020: 190-192):

• A Sogdian merchant called An Nyo ho 安如宝 (the same 'ethnic' family name as famous General An Luchan in the middle 8th-century Tang) came to Nara from together with the famous Chinese Buddhist Priest, Ganjin Wajo in the middle of the 8th century. An himself also became a Buddhist priest in Nara, and ascended to second-in-chief high-priest of the 'Chinese temple' (Tosho Daiji) in Nara.
• In 736, a Persian called Li Mitsuei (?) 李密翳 came to Japan, also as one of the entourage of Indian Priest from China. We know little about his identity in detail, though.
• In 2016, a wooden tablet (linked to the newspaper article in Japanese, with the picture of the found tablet) was excavated in Nara, and a name of Persian person is found on it: 破斯清通. 破斯 means 'the Persians', and the tablet also specifies the post of official he worked as. He was a specially appointed 4th rank secretary (special, but probably not so specially high-ranking official), so I suppose he could probably understand Japanese to some extent and worked together with his Japanese fellow officials.

It is known that some Sogdians as well as Percians lived either in their diaspora colony in Tang China or in Korean Balhae/ Balhai (698-926), and Moriyasu suggests that they were sometimes employed also as a members of diplomatic envoys to Japan or engaged in over-sea trading merchants in Japan (Moriyasu 2020: 191).

So, at least for the inhabitants of the capital, you would certainly be not the first foreigners with blue eyes and possibly bushy beards.

References (all are written in Japanese, sorry):

• HIRAMOTO, Yoshisuke. 'Secular Change of Estimated Stature of Japanese in Kanto District from the Prehistoric Age to the Present Day'. Journal of the Anthropological Society of Nippon 80-3 (1972): 221-36. https://doi.org/10.1537/ase1911.80.221.
• MORIYASU, Takao. Global History Seen from Silkroad (Original title: Siruku Rodo Sekai Shi). Tokyo: Kodan Sha, 2020.