There would have been few if any Indians setting out from their home specifically for Japan. Rather, it seems that people like Bodhisena came to Japan after initially setting out for China, which by the Tang dynasty had become a centre of Mahayana Buddhism with major monasteries and holy sites gaining fame throughout the Buddhist world. In particular Mt Wuati came to be known as the home of the Boddhisatva Mañjuśrī and was attracting Indian pilgrims by the end of the 7th century. The main route between China and India by then was through Southeast Asia, with a regular trade by sea developing during the Tang and a major centre of Buddhist learning growing up in Srivijaya, where Chinese monks like Yijing could mix with their Indian counterparts and presumably spread the fame of sites like Mt Wuati. At the same time, Japanese pilgrims were flocking into China both on their own initiative and on official missions, seeking to bring Buddhist learning back to Japan. Inevitably these travellers came into contact, and this is how the likes of Bodhisena could end up invited to Japan. And though Indians in Japan were certainly very rare, Bodhisena was not alone and was even accompanied by another Indian (or Malay, or Cham) on his way to Japan.

Tajihi no Hironari led the ninth embassy to Tang China in 733. While in China, in 742, [the Japanese Buddhist monks] Fusho and Eiyo met the famous [Chinese] monk Ganjin (Ch. Jianzhan) and invited him to come to Japan. Eiyo did not survive the trip back to Japan, but in 754, Fusho succeeded in returning to Japan with Ganjin. On its return to Japan in 736, the ninth embassy brought with it Bodhisena (Jp. Bodai, 704-760), the Indian or Malay musician Foche (dates unknown), and the Chinese monk Daoxuan (699-757), all of whom participated in celebrating the eye-opening of the great Buddha in Nara in 752.

Source: Charlotte von Verschuer (1999), Japan's Foreign Relations 600 to 1200 A.D.: A Translation from Zenrin Kokuhοki, Monumenta Nipponica, Vol. 54, No. 1, p. 23, footnote 46

By the fifth century the fringe of this Indian diaspora may have reached moder Korea. Of greater relevance, a few Indians seem to have even put ashore in Japan. The Nihon shoki records that in 654 four persons from Tokhara (Afghanistan) and a woman from Sravasti (northeast India) were blown by a storm to Hyuga, in southeast Kyushf. Writing in the nineteenth century, Aston dismissed this ancient Japanese record with the observation that "it is absurd to speak of natives of India being cast ashore" in Japan. In the light of the archeological and other evidence for a universalized Indic community extending throughout Southeast Asia in this period, however, it is not at all unbelievable that isolated Indians or other Indianized persons might have voyaged as far as the coasts of Japan, although the recorded Indian place-names may well be garbled ("exaggerated," or embellished, perhaps), and their numbers must have been few. There is also, moreover, the well documented case of the south Indian brahmin Bodhisena, who famously officiated at the ceremony "opening" the eyes of the Great Buddha at Nara in 752, and who arrived in Japan in 736 in the company of a Cham monk he had met "at sea."

Source: Charles Holcombe (1999), Trade-Buddhism: Maritime Trade, Immigration, and the Buddhist Landfall in Early Japan, Journal of the American Oriental Society, Vol. 119, No. 2, pp. 287-288.

On the other hand, there is little indication of the reverse journey from Japan to India being successfully made in these early centuries, even though the Japanese actually had a lot of incentive to visit the birthplace of Buddhism.

But travel to India bordered on fantasy. Although both Chonen and Eisai had considered making a pilgrimage to the birthland of the Buddha, nothing came of their plans. The well-known Chinese travelers to India - Fa-hsien (J. Hokken), 340?-420?; Hstian-tsang (J. Genjo), 600-664; and I-tsing (J. Gijo), 635-713 - provided their readers in both China and Japan not only with the most authoritative geographical information about the region, but also with a hearty respect for the rigors of the journey. It was difficult enough for the Japanese in the Kamakura period to reach neighboring China, let alone distant India. There are indications that an occasional Japanese found his way to India, and doubtless a few did, but there are no prominent examples on record... A curious case is Emperor Kammu's grandson, Prince Takaoka Ai, 799-865, who took the religious name Shinnyo Arn. In 864 he left Ch'ang-an for India via Canton; it is assumed that he died en route in 865, but he may have reached Singapore... According to Chinese sources, a Japanese called Kongo Sammai had traveled to Central India.

Source: Robert E. Morrell (1983), Passage to India Denied: Zeami's Kasuga Ryujin, Monumenta Nipponica, Vol. 37, No. 2, pp. 79-80.

Much later though, during the early modern period, many I assume reached India through Portuguese connections. In the 1620s some even vetured to Angkor Wat, which they interpreted as the legendary Jetavana-vihara founded in the Buddha's own time, leaving behind graffiti on the temple's walls. Maybe not relevant but it's a nice peice of trivia.

Great info thank you