I mentionned some elements of response in another question on Rome in the middle ages. The best illustration of the use of the ruins of the ancient Rome in the medieval city was made in the museum of the Crypta Balbi. This museum focus on the archaeology of one building block (and his theater) through time. Look here the finnish legend tells: imperial period, late antiquity, and probably early and late middle age. If I remember correctly from my visit there the dates should be: 1. around IInd century, 2. 6th-7th century, the later should be in the XIVth century but I don't remember if the third is in the XI or the XIIth century.

One of the main reason the decoration dissapeared was because marble can be melted to create lime, usefull to maintain and build buildings. The same finnish site that held the previous photo also show a picture of the reconstruction of the lime kiln wich functionned in the building during late antiquity. Those lime kiln are found in (almost?) every roman city in late antiquity. An innumarable number of statues dissapeared in those structures. For example the exceptionnal discoveries of statues in the Rhones River in Arles, France a few years ago were interpretted by the archaeologist as a deposit attenant to a bronze furnace. One day, probably during a flood the building collapsed into the river and the bronzes were preserved. The quest for raw materials also explain the holes at the intersections of blocks in many roman ruins because iron elements helped to maintain the blocks together, like on the ark of Janus Quadrifrons. On the same subject the Collosseum (and probably many other buildings) became quarries.

Another particularity of Rome is that without the Cloaca Maxima (the main sewer) the lower regions are regularly flooded bringing dirt and earth, enabling to cultivate some abandonned regions (the Capitolium became the hill of goats). That's why several meters of earth were removed when the Forum Romanum was excavated in the late XIXth-early XXth century. Some paintings of the XVIIth-XVIIIth century help the understand the result after the centuries. like this one or that one (when searching for such paintings, take note that some are pretexts to show a maximum of antiquities without making attention to the actual distributions of the ruins in the city). You can look at the difference by comparing the same monument like the arch of Septimius Severus in the XVIIIth century and today.

There are many many other examples but that's enough to cover most of the cases for Rome. With the Renaissance the popes (with some exceptions) forbade to destroy ancient monuments. The last transformation of importance came with Mussolini who wanted to show the might of the ancient Rome by excavating the Imperial Forums, razing the houses that were built upon them. Then he created the Via dell Impero (Road of the Empire) called today the Via de Fora Imperiali.

The process was quite the same in the other cities of the western roman empire. In Gaul a new defensive wall (often using old tombstones as foundations) were built within the ancient cities, showing the reduction of the population, it was also easier to protect.