When standing buildings were demolished, it was often very systematic, in order to recycle as many of the materials as possible. Large buildings like churches contained a lot of useful material: stone that had already been quarried and transported to the city, iron used to hold the stone blocks together, lead sheets on the roof, strong timber beams, etc. It was not uncommon to remove everything but a building's foundation and use it for the new structure. So, for example, the cathedral on Old Sarum was stripped down to its foundations, and some of the stone used to build the new cathedral and town in Salisbury (2 miles away). Conwy castle in Wales was built by Edward I by destroying a neighboring castle and moving the blocks a few miles to set up the new stronghold.

When buildings burned down or collapsed from neglect, their recycling might be a little more piecemeal (as opposed to official relocation projects, like when the church at Old Sarum chose to move its location) and continue for several centuries. Farmers were slowly recycling Hadrian's Wall up until the 19th century (if you ever get the chance to walk it, notice what all the sheep fences neighboring the wall are made from). This is why there aren't many heaps of stone from ruined Roman villas in England anymore - mostly recycled.

There were three primary methods of demolishing masonry buildings, each offering advantages and disadvantages. The first stage was to remove valuable materials: lead from the roof and windows, iron fittings, salvageable timberwork, saleable items like carved fireplaces, doors, tiles, panelling etc. It was not uncommon for large structural elements like complete roofs, staircases, arcades of pillars, decorative stonework of windows and doors, sculpture etc to be sold off and removed for reuse.

After this there was a choice. The quickest method, used particularly on castles that needed to be swiftly rendered indefensible, was to undermine foundations and structural supports and prop the building up on heavy timbers. The supports were then burned, or pulled out by teams of horses or oxen, after which the building would collapse. This method required skilled engineers to do and was dangerous, also the collapse tended to smash the masonry to bits or leave in huge lumps, limiting resale value. In the late mediaeval and early modern period gunpowder could be used to achieve the same results as mining, but it was very expensive and had the same disadvantages in terms of resale of masonry.

The second method was demolition by hand using picks, wedges etc. This was expensive, slow and required much work, but it did maximise the amount of salvaged materials that could be sold. To give an idea of what was involved. in 1648 the smallish castle of Montgomery in Powys, Wales, was ordered destroyed by Parliament. A team of 150 workers was needed, including miners, carpenters, masons and labourers. The work took from June to October and cost £675, around £1.5 million at 2014 prices. Much of the castle was reduced to ground floor level but large sections up to 50' high remain. Unfortunately we don't know what the profit from the sale of materials was, it is likely to have been substantial.

The third method was simple, and cost free, to let it be known that the building was available for quarrying. Cut stone and brick were valuable and desirable materials and the local populace would take them for their own purposes, in this way a massive building might vanish in the course of a few years or a few centuries, depending on local demand for building materials.

Source: the demolition accounts for Montgomery Castle are reproduced in The Decline of the Castle, M. W. Thompson, Cambridge University Press 1987

There was a wonderful gentleman named Fred Dibnah in the UK who practiced traditional demolition techniques right up into the 2000's. There are plenty of videos of him demolishing chimneys on youtube where you can see the undermining, the shoring up with timbers, and the collapsing with fire, enjoy!

https://www.youtube.com/results?search_query=fred+dibnah+demolition

There's an interesting point in this video where he's being showed photographs of his predecessors (2m in):

http://youtu.be/0L1WOnR2KBY?t=2m

You may also find "slighting" interesting. It's the practice of rendering a castle unsuitable for use as a fortification, but it mainly came about because even with dynamite, straight up demolishing a castle is quite simply bloody hard.