For a start, here's another similar answer I've written before: https://www.reddit.com/r/AskHistorians/comments/t2tjex/what_was_the_influence_of_china_on_the_indian/

You have 2 premises in this question. One is that Indian and Chinese Empires were bordering, and the second is that their interactions were rare and small-scale. I'll give a small overview of Indo-Chinese relations in history to disprove this.

Your first premise is that Indian and Chinese 'empires' would have bordered. However, you are going by the modern political boundaries between the 2 countries. While India and China have one of the world's longest borders, most of it is along high-altitude wasteland (Aksai Chin), or massive mountains (Tibet). Indeed, these were scarcely populated regions. Another thing is that, Tibet has only been under formal, central Chinese control since 1951, before which the Tibetans were vassals who rarely had to interact with Beijing. So for most of history, Indian and Chinese civilizations were apart.

Going overland from India to China or vice versa was a long and time-consuming affair. One would have to either go through the high mountains of Tibet, through the dense hilly forests of Yunnan, Myanmar and Manipur, or the dry steppes of Central Asia. All these areas had the threat of hostile tribals, and the writ of Indian or Chinese polities rarely extended there. As a result, they were rarely travelled, except by monks (such as Faxian and Xuanzang, who are the monks you mentioned, and who I'll return to later). The overland routes wouldn't be much use to trade goods, and so were effectively useless, except for monks, who travelled alone, and went to see Buddhist sites on the overland routes.

That left the sea. Indians were prolific traders since the Harappan era, and polities all over the coasts traded with the Arabs, Egyptians, and even the Romans. On the other way, trade went on with Southeast Asians and the Chinese.

These links are ancient. There is some evidence of distinctively Harappan decorated beads being found in some Zhou dynasty sites, and centuries later the author Chanakya calls silk cloth 'cinamsuka' (Chinese cloth) in his Arthashastra. In the Tamil lands, coast cities have turned up Chinese coinage. Similarly on the Chinese side, a land called Shendu (probably from Sindhu) was mentioned, likely meaning India.

Interactions picked up speed with the arrival of Buddhism, which was a more tangible link between India and China. Monks from India went to China as teachers (possibly including Bodhidharma, the founder of Zen Buddhism), and Chinese monks came to learn and take Pali texts to China. Xuanzang and Faxian travelled for this purpose, and their records are an excellent account of the state of India at the time of their visits. The most interesting interaction of this time was Wang Xuance's defeat of a pretender king named A-luo-na-shun, which I've written about in the previous answer I linked.

From this era, there were also small communities of Indian traders in major Chinese ports. Quanzhou has some concrete evidence of this, being home to a bilingual Tamil-Chinese inscription dedicated to Lord Shiva, with a distinctly South Indian carving of the god. The Kaiyuan Buddhist temple in the city was once a Hindu temple used by the local Indians, and even today some Chola carvings are found in the surrounding areas.

The Chola dynasty, who I just mentioned, was a major thalassocracy from the 10th century onwards. Under Rajendra I, they turned the Sriwijaya kingdom of Indonesia into their vassals, bringing the Indian and Chinese worlds much closer. The Cholas were said to have maintained good relations with the Chinese, regularly sending embassies uptil their decline in the 1240s.

In 1405, Zheng He set out on his first treasure voyages, during which he touched upon India's east coast, in the region of Kerala. I have mentioned in the linked answer how he turned the state of Kochi into a Ming vassal. Kerala in the medieval era was most important for trade, and as a result, it was very cosmopolitan, with Arabs, Jews, Gujaratis and Chinese settling there.

One of the most famous Chinese people in Kerala was Chinali (means Ali from China). Kunjali Marakkar was the title given to the admirals of the Kozhikode ruler, the Saamoothiri. The fourth Kunjali Marakkar is seen as a great hero for his opposition to the Portuguese in the 1500s. He is said to have rescued Chinali from a Portuguese slave ship and adopted him. Chinali quickly became one of his favourites. This person's existence is confirmed by Portuguese historians, who called him a fanatical Muslim who delighted in killing Christians. The allegation is likely untrue, not least because Kerala has a venerable Christian community, who Chinali would have helped protect; the Portuguese often viewed any opposition to their brutal tactics as oppression against 'Christians'.

Another interesting interaction was a one-year war fought between the Sikh Empire's vassal, Gulab Singh, and the Qing Chinese forces, over control of far western Tibet in 1841-2. Nothing much came of this war, however, since a treaty was signed maintaining the status quo, and both the empires faced huge crises that led them to withdraw (The Sikhs faced rising English aggression, while the Chinese were occupied with the Opium War.)

In the era of the British Raj, the British administration of India often came into contact with the Chinese during the 'Great Game', where they battled for influence of Central Asia against Russia. Further, many Chinese settled in Kolkata during the Raj, and the city has a 'Chinatown' to this day. However, after this point I am not too much of an expert, so I'll leave off here.

TL;DR, Indian and Chinese empires didn't border for most of their history, but they had long and rich interactions.

More can of course be said for other periods, but I discuss the Qing context (drawing primarily from Matthew Mosca's excellent From Frontier Policy to Foreign Policy) in this answer.