To be clear, this is not a leading question or a question that seeks to call into doubt how old Hinduism is as a practise. I am just curious as to why the oldest recognised Hindu temple (Mundeshwari) only dates back to around 635 CE, Was temple construction not a thing in Hinduism before that? Were they torn down or destroyed? Thanks!
Firstly, it should be noted that the Mundeshwari temple is only one of the oldest recognised functioning Hindu temples. The remains of many older cultic structures have been found that date back to before the 1st century CE, but they are admittedly few in number compared to what we might expect of an ostensibly 4000 year old religion. If there were older temples, even if they were later destroyed, we would still be able to find their foundations, so they most likely simply weren't built.
It is necessary to recognise, in order to explain why recognisably Hindu temple structures don't appear until quite late, that to refer to the Indian religion that existed 4000 years ago as "Hinduism" is quite anachronistic. The modern religion we call Hinduism is really best characterised as a family of religions that share a common descent from ancient Vedic and non-Vedic Indian religious traditions. The claim that Hinduism is 4000 years old is derived from the fact that the earliest text recognised by all orthodox Hindus as canonical, the Rigveda, was, according to Dr. Gavin Flood, probably composed around 1000-1500 BCE, with the earliest estimates going back to 1900 BCE. But the religion described in the Vedas has relatively little in common with the religion(s) we now call Hinduism. Most of the rites, sacrifices and ritual practices are no longer practiced by most modern Hindus, or at least are no longer as central.
The primary deities of the modern Hindu pantheon, Vishnu and Shiva, are either scarcely mentioned or absent entirely from the Vedas, while the most important Vedic gods, such as Indra, Agni, and Dyaus Pitr, have since been reduced to minor deities or forgotten. For this reason the religion of ancient India is usually referred to by scholars as the Vedic religion. The Vedas primarily emphasise ritual propriety and there is little in the way of the personal devotion or bhakti that would later become predominant. Because of this there was also little need for religious art or idols of gods to direct devotion towards, which (with the exception of the aniconic Lingam) would only appear far later, and therefore little need for elaborate temple structures in which to house them. It is possible that earlier idols existed, but if they did they were not carved in stone and have not survived.
It is only during the Gupta dynasty around the 5th century CE that we begin to see large scale construction of what we would later come to recognise as Hindu temples and that the foundations of all subsequent Hindu architecture were laid. Non-coincidentally, this is also around the period when the modern sectarian divisions in Hinduism, between Vaishnavism and Shaivism, were beginning to crystalise, and when the Puranas, devotional poems to Hindu gods, were written. It is also most likely not coincidental that the Gupta dynasty, while mostly Vaishnavite, were tolerant of and sometimes promoted Buddhism, which had a much older and more established architectural and artistic tradition. This Buddhist tradition, in turn, largely originated from or was at least heavily influenced by Indo-Greek art and architecture, whose very image-centric culture created the first sculptural depictions we have of the Buddha. So it seems, from what we are able to determine, that art and architecture were not originally central to Vedic religion and that its centrality to later Hinduism is most likely, directly or indirectly, a Buddhist influence, which was itself most likely a product of Hellenic influence.
Edit: corrected grammatical errors
I'm sure a more precise answer to your question may appear soon, but I believe this older answer of mine might clear out a few problems you seem to have with your framework.
In this answer I have tried to establish a decent time line and set of belief systems which might allow us to seperate what we consider Puranic Hinduism, or the Hinduism that we know and practice today (with obvious alterations and differences between the former and the latter, in rituals, practices and beliefs occurring due to external and internal factors in the last 2000 years) and Vedic Brahminism.
This separation and understanding it's importance is necessary, since otherwise, we may include the former as part of our conception of the latter and proclaim practices, rituals and beliefs we understand to be common today, to have been similarly part of the belief system which was arguably a precursor to Hinduism, or rather the phenomenon which through changes and evolutions over time, became Hinduism as we know it today, to have been one and the same.
To quote my own concluding paragraph from the previous answer :
We therefore arrive at some fruitful conclusions. Hinduism and it's constituent components have been evolving throughout history while certainly Early Vedic religion and practices have marked differences when compare to Later Vedic and Classical practices, we find that as early as roughly the c.500 BCE to 400 BCE, the constituent components of Hinduism as we understand them today, began to emerge in their rudimentary and developing forms. In many ways, the Brahmanism of the Later Vedic Age may be seen as an earlier form or Hinduism as we understand it today. Now whether Hinduism is one of the oldest religions in the world may still be contested, but I hope this answer gives you some perspective into understanding how old some of the practices that we consider to be important components of Hinduism are. It should also be mentioned that this exercise fails to take account of those belief systems that have not been recorded in such ways that their information may be handed down to us. Such belief systems as may have existed during the palaeolithic, mesolithic or neolithic periods which have simply been lost to time.
I shall also add my sources here :
SOURCES
Asko Parapola The Roots of Hinduism : The Early Aryans and the Indus Civilisation
Upinder Singh A History of Ancient and Early Medieval India from the Stone Age to the 12th century
Romila Thapar Early India from the Origins to AD 1300
RS Sharma India's Ancient Past
Satish Chandra A History of Medieval India
Gavin Flood, The Blackwell Companion to Hinduism