It was made of just the same stuff that's left there today! And it isn't sandstone, it's mud, a mixture of sand and clay that can either be formed into sun-dried bricks (which are known as adobe), pooled in a mold to form a wall (in which case it is tapia or tapial and likely has small stones embedded in it to help keep its structure), and the basic everyday commoner houses could be of wattle-and-daub construction, which uses canes of various sorts and is sometimes called quincha architecture. You will also find posts made of algarrobo wood, which is a very hard wood that preserves very well in the desert environment (we've found some on site that is probably 1500 years old and was in such good shape that local residents took it to use in their house), and stones could also be used for things like thresholds through doors.

I have never excavated or really studied Chan Chan or any site of its time period, but the interesting thing is that the way of making walls on the north coast of Peru hasn't changed much in thousands of years. Poorer houses today are still often made of adobe and building with similar bricks goes back to sites like Caral some 3000 - 4000 years earlier than Chan Chan. So I am definitely no expert in Chimú architecture and I do not have first-hand experience with excavating these buildings (I've just done the Chan Chan tour a few times and surveyed some other contemporary sites), but this architecture resembles much earlier time periods and is quite comparable.

I have noticed in the Virú Valley, which is just south of the Moche Valley (where Chan Chan is located) that residential buildings on the rocky hills and quebradas (ravines and alluvial fans full of boulders) were built with stone foundations and likely had adobe and/or cane structures above the foundations. In the valley floors just a few hundred meters away, everything was adobe or tapia for the foundations. That isn't surprising (people use was is readily available to them), but it also shows that the readily available rock was not considered necessary for building a good building. Adobe probably was, however (at least for the big public buildings) and you could also plaster adobe easily and paint the walls, or make the sorts of molded marine themes that you see at Chan Chan (birds, fish, fish nets, etc.) I suspect that the same pattern is in the Moche Valley, but I have not done very extensive survey there.

Preservation on the north coast of Peru is really really good if something ends up buried (including if it is placed within a wall), so if it is made of wood or cane we have a good chance of finding it or finding enough traces to know what was there. So what you see now is pretty much what was there, just more decayed. The thing is, much as the north coast is arid and usually doesn't rain a lot, it is pretty humid, there are regular strong winds, and during an El Niño year it will rain a ton. Unbaked mud bricks or pooled mud doesn't hold up very well under those conditions, and walls need to be repaired or replaced fairly regularly. So when Chan Chan went into decline after Spanish settlement and disease, the walls began to erode into the shapes they are now in. We also generally think that each of the big compounds at Chan Chan were palaces and administrative compounds built by one ruler and abandoned after their death, so these might not have been maintained even before the city was depopulated.

Unfortunately I can't spend a lot of time on this right now and this is mostly from memory, but there are some great sources on architecture at Chan Chan and throughout the region, although most don't address your question explicitly. I also found a few promising sources that I am not able to read right now, but that look useful. Search for these through scholar.google.com and you may be able to access them, especially if you have an academia.edu or researchgate account:

Moseley and his colleagues did a lot of work at Chan Chan in the 70s and their papers still form the bulk of what we know about the city. This is Moseley's general textbook about the archaeology of the Andes, but it is still a good introduction to the region and he talks about architecture on the north coast:

Moseley, M. E. (2001). The Incas and Their Ancestors: The Archaeology of Peru (Revised Edition).

Topic, J. R. (2003). From stewards to bureaucrats: architecture and information flow at Chan Chan, Peru. Latin American Antiquity, 243-274.

Moore, J. D. (1992). Pattern and meaning in prehistoric Peruvian architecture: The architecture of social control in the Chimu state. Latin American Antiquity, 3(2), 95-113.

This one looks really useful and it should be open to anyone:

Tsai, H. I. (2012). Adobe bricks and labor organization on the North Coast of Peru. Andean Past, 10(1), 9.

Kolata, A. L. (1990). The urban concept of Chan Chan. The Northern Dynasties: Kingship and Statecraft in Chimor, Dumbarton Oaks, Washington, DC, 107-144.

Camilloni, H. R. (2003, January). Quincha architecture: The development of an antiseismic structural system in seventeenth century Lima. In First International Congress on Construction History.