These cities in the middle ages were already established and densely populated by the time the idea of city wide sanitation became a thing.
Were whole streets just demolished to lay underground tunnels and pipes?
Edit. Thanks for the answers everyone! This question has been bugging me for ages.
For a long time in London, sewers were above ground and ran along outside houses to cesspits. If you were close enough to the rivers the ran nearby, such as the Fleet, Tyburn or Westbourne, it ran in to these and in to the Thames. As years grew by and London got busier, these rivers were enclosed and built over bit by bit.
In 1848, James Newlands started the construction of the world's first integrated sewer system in Liverpool, mostly using cut and cover techniques. After making an incredibly large amount of surveys, he created one of the most detailed height maps at the time of Liverpool, which enabled him to decide the most efficient routes of sewers.
When it comes to London, Joseph Bazalgette is considered to be the grandfather of sewers, although James Newlands should take that regardless. Bazalgette started a mammoth amount of works to overcome the "Great Stink" of 1858. If Parliament hadn't have been affected, who knows where we'd be today.
What came in the following decades was a marvel of engineering. What we today call the "Embankment" in London, (Victoria Embankment/Chelsea Embankment) are actually pieces of the River Thames that were reclaimed under Bazalgette. The river was pushed back and the land was dug down. In terms of what happened in London, the sewers were then built by hand completely by brick. At the same time though, the Underground was laid along the same land that had been reclaimed. This was all then covered over, roads put on and then forgotten about. Bazalgette also fully enclosed the River Fleet and Tyburn with brick, and connected them to the new primary sewers.
So to answer your question, in London, after building over the rivers and streams that flowed directly in to the Thames, it was a case of cut and cover techniques.
Musuem of London have a great article and also an excellent exhibition about it. In modern times, we are currently building a 25km "Super Sewer", to help alleviate centuries of backlog and to keep effluent away from the Thames and directed to treatment facilities. This time though, instead of cut and cover, we now use tunnel boring machines, such as those used to create the Crossrail tunnels. Thames Tideway Project is a project that is not really spoken about but is arguably more important than Crossrail.
In the UK, Channel 5 did a fantastic series that covered this, Episode 2 of "How the Victorians Built Britain", 'Saving the Nation's Health'. Worth a watch if you can find it online.
Follow-up: In the cities where buildings were lifted or streets excavated, how was it funded? Was it a significant monetary cost?