Was house-to-house fighting common before the 20th century? And by house to house I mean clearing of houses one by one in the city by ground troops, or did most big battles for cities take place outside of the city limits?
If you mean fighting involving firearms, I do think that house-to-house or urban warfare would have been more recurrent in the XIXth century than before. I think I heard about a house-to-house combat during the Napoleonic Wars, however I can't confirm it.
There is one example I can think of though, that took place in September 1870 during the Franco-Prussian War.
The Battle of Bazeilles in the Ardennes ( France ) saw heavy fighting between Bavarian troops and French Imperial soldiers for the control of streets and even houses in the village of Bazeilles. Here is a photograph taken after the end of the battle showing the streets.
Here is a translation ( that I made so please excuse possible mistakes ) of the official report of the battle ( called Bazeilles et Sedan : essais critiques sur les opérations de l’armée de Chalons… published in 1871 ) :
" One against ten, the marsouins ( name given to the French marine infantry ), despite the shells crushing them and the fires burning and suffocating them defended every inch of every street, every house and every wall. They ceded terrain very slowly and only after inflicted the enemy heavy casualties. [...] "
This battle also inspired De Neuville's The Last Cartridges painting and the same name 1897 movie by George Méliès.
Source : The website of the French Marine Troops has a transcription of official report of the battle of Bazailles.
I also had the chance to visit this town which has an incredibly moving museum called The House of the Last Cartridge in the inn where the fighting depicted in De Neuville's painting took place. There is a room that has not been changed since 1870 where you can still see bullet holes in the walls. Here is a picture of the room today ( I let you compare it with the painting ).
From what I remember of the visit, the guide explained that this sort of fighting was relatively new for the French and German Infantry and mostly due to new firearms. For example, the French Chassepot rifle ( 1866 ) was a breech-loading firearm which allowed soldiers to reload in various positions and no longer only while standing-up. This let soldiers to fire while lying down for example. These rifles like the Chassepot were also much more precise than what they used before which slowly made line infantry battle becoming obsolete.
There certainly has been similar battles earlier than 1870 but this is the earliest example that I know of ( which is a regular battle and not fighting like in riots or revolutions ).
Do you mean urban combat excluding rebellian and sacking/looting?
Are you thinking more 1600s 1700s 1800s or more like 300s or 600s or 1200s?
House-to-house fighting occurred only to a very limited extent, though I'm not aware of any fighting in what we would recognize as a city. Mostly it would be fighting inside towns and villages on the edges of the battlefield.
Principally this is because prior to the 20th century commanders really had to see the action in order to exercise control over their forces and react to enemy movements. Once you've entered a large town or city you lose most of that control. To some extent that's even true today.
Also, prior to the advent of accurate breach-loading firearms there was no real advantage to defending an urban area (as opposed to a fort or fortified stone building). You can't really hit an enemy with a sword from inside a house and non-rifled muzzle-loading muskets relied on massed fires for their effect.
Some examples I'm aware of:
Fighting in the town of Fredericksburg between Union and Confederate forces during the Battle of Fredericksburg December 1862.
http://en.wikipedia.org/wiki/Battle_of_Fredericksburg
During the Battle of Waterloo, June 1815, arriving Prussian troops had to fight through the villages of Frichermont, Papelotte and Plancenoit in their assault on the French right wing.
http://en.wikipedia.org/wiki/Battle_of_Waterloo#Arrival_of_the_Prussian_IV_Corps:_Plancenoit
Fierce fighting did take place around the three villages of Lutzingen, Oberglauheim and Blenheim during the Battle of Blenheim, August 1704.
