We actually have some decent contemporaneous accounts of Chinese pilgrims visiting India during Buddhism's stable phase and then further accounts during the terminal phase.

Faxian describes his experiences in India in his text Foguoji (A Record of Buddhistic Kingdoms). He departed China in 399 as part of a pilgrimage mission. He observes that there were apparently thriving monastic communities at Kusinagar (where the Buddha attained parinibbana) and other Buddhist sites. He observes that at this time Kapilavastu (Buddha's home city) and Lumbini (his birthplace) are desolate and dangerous places. If you visit Kapilavastu today I would say it is pretty similar to when Faxian visited. All that is left are some crumbling walls. We also see evidence in his records of Buddhist sites that were in a state of degradation - he describes how a holy site built over a place in Rajgir where Buddha preached Dharma was basically destroyed.

Faxian describes his visit to the temple site in Bodh Gaya where the Buddha attained enlightenment. He notes that the temple is well looked after by Buddhist families and that monks who live in the area observe the precepts carefully. (Eventually this site is reclaimed by the Hindus - when it is basically discovered again by Buddhist missionaries in the 19th ce it is heavily overgrown and the Buddha image has been replaced with Hindu iconography. Anagarika Dharmapala - a Sinhalese Buddhist missionary - describes his experiences of this rediscovery in his biographical writings).

Next we have Xuanzang who visits India in the 630s. He recorded his observations in the text The Great tang Records of the Western Regions. We learn from him that, at this stage, the situation was quite perilous and we get this very clearly from his writings. He reports that when he visited Jetavena it was basically in ruin. This was previously a major Buddhist site. Kapilavastu is predictably empty. Interestingly his visit to Kusinagar reveals an empty city with few inhabitants - just a few hundred years prior Faxian reports that there was a thriving monastic community. Again, at the city of Vaisali - previously having a community of monks - he finds "very few monks." When he does find monastic communities he describes the conditions in negative terms - very often the monks are deprived and without lay support. The temples are usually dilapidated.

Xuanzang does complain about the fact that many of the monastic communities are Hiniyana and notes especially that they eat "the three kinds of pure meat" which he regards as evidence of not elevating themselves to a higher interpretation of scripture (i.e. the type of Buddhism he is familiar with in China). It is a subtle way of suggesting that they are pious but misguided.

In general, we get a sense from Xuanzang that Buddhism in India is in a pitiable state at this time. So in answer to your question, yes, there are historical accounts of Buddhism where we get a direct description of the condition of Buddhism in India and the impressions of the Buddhist pilgrims.

It is very interesting to note just how rapidly (comparatively speaking) things had degraded between Faxian's initial visit and Xuanzang's visit a few hundred years later. By the 9th ce Buddhism is all but completely destroyed in India. In fact many of the Indian Buddhist texts that we currently have only exist because they were kept in Tibet and translated to Tibetan during the initial dissemination of Buddhism in Tibet. Likewise, we also have surviving manuscripts kept in Chinese recensions. But major Indian Buddhist institutions of learning such as Nalanda were basically annihilated and the texts there lost.

There are many reasons for the decline of Buddhism in India. Hindu revivalism is one cause - Buddhism just ultimately became obviated by this more popular religion. Another important factor was a lack of state sponsorship - Buddhism originally thrived under state patronage and there was a long tradition of Indian rulers supporting many different religions, including Buddhism. But it seems that at some point this support ended. We also have to consider the fragmentation of the Buddhist tradition in India. After the Buddha's death the sangha broke up into many competing factions. This rivalry was probably not very healthy for Buddhism's longevity in its home nation as resources were stretched thin. Probably the least important aspect is the foreign invasion factor - while Muslim incursions into India was an issue for Buddhism (and all the native religions), Buddhism was virtually already destroyed.