One thing to note is that wealthy and important people typically built out of wood. Like you mentioned, temples and palaces were built out of wood, as were residences of the wealthy. By contrast, vernacular architecture in many places were built out of brick and stone.

One reason is that wooden buildings have advantages in earthquakes (which is why California also builds out of wood). Wood has more flexibility, and even when it does fall, it does not tumble down like rock constructions tend to do. Indeed, Chinese architecture had earthquake resistance in mind. Take a look at this model of the Linde Hall (Tang Dynasty, part of the Daming Palace). Note how the palace made use of interlocking beams, which makes it less likely to fall apart in the event of an earthquake. This video (in Chinese), shows the construction of a Liao era temple which also had the same principle in mind (also take a look at 6:27). Of course, earthquakes were not the main reason, but it was a major contributing factor.

Also, given the complex geography of China, wood was much easier to transport than stone. Logs could be cut down from forests and then floated down rivers, streams, and canals without much human involvement. When it reaches land, it could be rolled to its destination. Wood was also more abundant in China and easier to process.